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CURRENT  PRACTICE  AND  THEORETICAL  FOUNDATIONS  OF  THE  AFTER  ACTION 
REVIEW 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  After  Action  Review  (AAR)  has  been  the  foundation  of  U.S.  Army  training  for 
decades.  Although  leaders  and  Soldiers  agree  on  the  importance  of  AARs  for  training,  little  is 
known  about  why,  from  a  learning  theory  standpoint,  AARs  enhance  training,  or  how  AARs  are 
currently  practiced  in  the  US  Army.  Therefore,  ARI  decided  to  sponsor  research  to  (a)  identify 
the  theoretical  foundations  of  the  AAR  from  learning  theory,  (b)  determine  how  AAR  facilitators 
are  trained,  (c)  establish  how  AARs  are  currently  conducted  in  the  US  Army,  and  finally,  (d) 
seek  out  ways  AARs  might  be  improved. 

Procedure: 


We  investigated  theoretical  foundations,  doctrinal  guidance,  and  the  perceptions  of  U.S. 
Army  personnel  regarding  the  importance  of  AAR  to  training  effectiveness.  The  latter  included 
what  approaches  work  best,  and  how  the  AAR  is  practiced  today.  First,  a  literature  review  was 
conducted  to  identify  what  the  learning  research  had  to  say  about  how  AARs  influence  learning 
(see  ARI  Research  Note  201 1-X).  The  research  team  observed  AARs  being  executed  at  the 
National  Training  Center  (NTC)  and  within  the  Battle  Command  Training  Program  (BCTP) 
followed  by  interviews  of  their  Observer  Controllers  (O/Cs).  Additional  observations  of  AAR 
practice  at  Fort  Stewart,  ga  and  Fort  Polk,  LA  followed.  An  online  survey  deployed  via  U.S. 
Army  Knowledge  Online  (AKO)  resulted  in  901  responses  spread  across  all  ranks  from  General 
Officer  to  specialist  and  all  experience  levels. 

Findings: 


The  key  findings  from  this  research  show  that  the  AAR  is  a  fundamental  part  of  the  U.S. 
Army  training  culture  with  recognized  value  at  the  trainer  and  trainee  level.  However,  practice 
of  the  AAR  varies  across  the  U.S.  Army,  and  among  individual  practitioners  more  so  than 
between  training  programs  or  organizations.  Survey  respondents  were  nearly  universally 
supportive  of  the  AAR  as  a  key  contributor  to  training  effectiveness,  but  also  provided  us 
additional  insight  based  on  their  experiences  as  to  what  works  best.  Previous  research  by 
members  of  the  research  team  had  resulted  in  a  proposed  Integrated  Theory  of  AAR  (ITAAR) 
model.  During  this  research  we  reevaluated  the  completeness  of  that  model  and  identified 
enhancements  to  it.  Military  research  regarding  the  AAR  has  been  led  by  a  few  key  individuals 
and  organizations  that  primarily  focused  on  technology-supported  training  environments  and 
how  to  best  support  or  conduct  AARs  therein.  AAR  guidance  to  U.S.  Army  personnel  or 
doctrine  is  several  decades  old  but  not  necessarily  out  of  date  as  the  principles  are,  for  the  most 
part,  timeless. 
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Utilization  and  Dissemination  of  Findings: 


The  findings  indicate  a  need  for  training  on  the  conduct  of  AARs  within  officer  and 
professional  education  programs.  Such  training  should  familiarize  students  with  the  underlying 
theoretical  and  learning  principles  which  inform  the  process  of  AAR,  provide  them  techniques 
and  best  practice  for  facilitating  an  AAR,  and  provide  realistic  opportunities  to  practice  those 
techniques.  Future  research  may  be  warranted  to  develop  a  standard  curriculum  for  Army 
schools  and  centers.  Such  an  effort  would  investigate  differences  among  AAR  system  - 
capabilities,  AAR  processes,  and  best  practices  to  support  different  training  environments  (e.g., 
Live,  Virtual,  Constructive,  Gaming).  Multidisciplinary  training  on  how  to  execute  an  AAR, 
sponsored  by  TRADOC,  is  recommended.  A  need  also  exists  for  the  development  of  specific 
learning  materials  to  support  operational  unit  training  and  as  a  reference  for  use  by  practitioners 
at  all  levels.  Whether  that  supporting  material  is  in  the  form  of  a  text,  guidebook,  or  set  of 
interactive  training  aids  remains  to  be  determined. 
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CURRENT  PRACTICE  AND  THEORETICAL  FOUNDATIONS  OF  THE  AFTER  ACTION 
REVIEW 


Background 

Cognitive  and  Learning  Sciences  Research  Summary 

Researchers  have  long  acknowledged  provision  of  feedback  as  essential  for  efficient 
learning  (c.f.,  Mory,  1992).  Yet,  the  methods  available  for  administering  feedback  vary  widely, 
as  does  the  consistency  with  which  they  are  used  by  trainers  to  accomplish  feedback 
administration.  Unfortunately,  failures  to  provide  effective  feedback  have  led  to  poor  knowledge 
retention,  poor  knowledge  transfer,  and  a  tendency  among  some  to  discount  the  importance  of 
the  feedback  process.  Some  of  these  failures  are  caused  by  difficulties  capturing  meaningful 
data.  More  often,  poor  feedback  stems  from  the  non-intuitive  design  of  feedback  sessions  and 
variability  in  feedback  administration.  Consequently,  trainees  who  could  benefit  from  proper 
feedback  administration  may  adopt  a  cavalier  or  skeptical  opinion  of  the  importance  of  training 
feedback.  The  complexity  of  modem  military  missions  and  the  necessity  for  seamless 
interaction  of  Soldier  team  members  requires  that  feedback  be  consistent  and  effective,  and  that 
it  be  delivered  to  ensure  maximal  retention  and  task  transfer  (Baird,  Holland,  &  Deacon,  1999). 
For  a  more  thorough  review  of  the  literature  in  this  area,  see  Bliss,  Minnis,  Wilkinson, 

Mastaglio,  &  Barnett  (in  preparation). 

Description  of  After  Action  Reviews 

An  after  action  review  (AAR)  is  a  retrospective  session  that  allows  trainees  to  review 
prior  training  performance.  AAR  sessions  are  not  intended  to  be  critical  evaluations  of 
performance  so  much  as  collaborative  meetings  held  to  enhance  learning.  Ideally,  AARs  should 
be  conducted  to  focus  on  what  was  planned  to  occur  during  training,  what  actually  did  occur 
during  training,  why  events  unfolded  as  they  did,  and  what  should  be  modified  during 
subsequent  training  or  performance  sessions.  It  is  important  to  emphasize  that  AARs  are 
generally  conducted  at  the  team  level.  Though  individual  AARs  are  possible,  they  are  typically 
introspective  and  personal.  Optimally,  AAR  sessions  should  be  held  as  soon  after  training  as  is 
practical. 

How  a  typical  AAR  is  conducted.  An  AAR  is  typically  led  by  a  facilitator,  sometimes 
the  unit  leader,  but  for  more  formal  AARs  it  can  be  an  Observer/Controller  (O/C)  or  other 
experienced  trainer.  The  facilitator  is  ideally  a  training  expert  who  is  also  familiar  with  the  goals 
of  the  training  session.  Facilitators  are  typically  non-commissioned  officers  or  officers  with 
considerable  experience. 

The  facilitator  typically  starts  the  AAR  by  reviewing  the  mission  and  training  objectives. 
The  facilitator  then  encourages  the  training  audience  to  discuss  what  happened  at  specific  points 
during  the  exercise,  including  not  only  the  actions  of  the  unit,  but  how  effective  those  actions 
were  to  meet  mission  objectives.  The  members  of  the  unit  are  expected  to  reflect  upon  their 
performance,  consider  whether  their  performance  met  U.S.  Army  standards,  and,  if  not,  reflect 
on  how  they  could  improve  their  performance  in  the  future.  The  facilitator  chooses  what  points 
to  discuss  and  provides  “ground  truth”  when  information  is  missing  or  misunderstandings  occur, 
but  individual  unit  members  are  expected  to  evaluate  their  own  performance. 
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AARs  in  Practice 

There  is  considerable  variability  with  regard  to  how  military  AARs  are  conducted  during 
unit  training.  This  variability  most  often  reflects  the  complexity  of  the  training  situation.  Many 
training  sessions  (particularly  at  major  facilities  such  as  the  National  Training  Center  (NTC)  at 
Fort  Irwin,  CA)  may  stretch  for  two  weeks  or  more  with  Soldiers  engaged  in  simulated  combat 
against  thinking,  flexible,  creative  opponents.  Over  that  period,  there  are  several  types  of 
feedback  sessions  that  trainers  call  AARs,  ranging  from  informal  “hotwashes”  to  more  formal 
collaborative  sessions.  For  this  research,  we  claim  that  feedback  sessions  which  use  a  specific 
approach  that  is  based  on  facilitation  of  a  review  and  feedback  process  primarily  driven  by  the 
training  audience  are  AARs.  (See  Appendix  B  for  a  full  review  of  AARs  in  practice.) 

An  important  variable  that  impacts  AAR  implementation  is  the  nature  of  AAR  facilitators 
(Keene,  1994).  Each  one  may  use  a  different  philosophy  about  how  best  to  approach  the  team, 
and  may  espouse  a  different  interaction  style.  Different  facilitators  may  be  more  or  less  effective 
at  stimulating  discussion,  leading  participants,  structuring  the  lessons,  reflecting  on  planned 
goals,  and  synthesizing  the  lessons  learned  from  the  AAR.  Trainees  themselves  also  constitute  a 
source  of  variability.  Some  groups  function  cohesively  with  a  clear  leader.  Other  groups  may 
contain  unfamiliar  members,  may  have  a  less  effective  leader,  or  may  suffer  from  internal  strife 
or  conflict.  These  issues  potentially  cause  AARs  to  be  inconsistent  and  their  outcomes  can 
therefore  be  unpredictable. 

Facilitators  should  have  a  clear  understanding  of  the  underlying  principles  of  the  AAR 
process.  Failure  to  understand  these  principles,  for  example,  the  difference  between  an  AAR  and 
a  critique,  can  rob  the  review  of  its  effectiveness.  Early  attempts  to  transition  the  AAR  concept 
to  the  corporate  world  had  limited  success,  possibly  because  of  a  misunderstanding  of  AAR 
principles.  Peter  Senge  (2001)  remarks: 

The  Army's  After  Action  Review  (AAR)  is  arguably  one  of  the  most  successful 
organizational  learning  methods  yet  devised.  Yet,  most  every  corporate  effort  to 
graft  this  truly  innovative  practice  into  their  culture  has  failed  because,  again  and 
again,  people  reduce  the  living  practice  of  AAR's  to  a  sterile  technique. 

This  suggests  that  the  AAR  technique  should  be  more  than  simply  a  static  process  that  is 
transferable  across  organizations  and  contexts.  Rather  it  should  be  tailored  to  the  needs  of  a 
particular  context  and  requires  a  trainer  who  understands  the  underlying  principles  of  an  AAR, 
and  is  adept  at  facilitating  the  group  introspection  and  communication  that  allows  a  team  to 
engage  in  self-learning  and  self-correction. 

A  second  issue  with  current  AAR  practices  is  that  they  are  frequently  defined  by  practical 
constraints,  rather  than  driven  by  key  findings  from  behavioral  researchers.  AARs  have  been 
referred  to  as  a  self-contained  substantiation  of  a  broader  knowledge  management  program. 

They  are  designed  with  the  overriding  purpose  of  enhancing  knowledge.  However,  to  realize  the 
benefits  possible  from  AARs,  we  need  to  review  the  publications  of  researchers  who  have 
studied  the  learning  process.  Scientists  have  published  volumes  devoted  to  curriculum  design, 
practice,  feedback  and  other  theoretical  concerns.  Yet,  too  often  the  implementation  of  AARs 
caters  to  time  constraints,  leadership  constraints,  labor  constraints,  and  monetary  constraints.  As 
a  result,  the  theoretical  value  of  the  technique  is  distilled  or  eliminated  altogether.  To  illustrate 
such  this  challenge,  DeGrosky  (2005)  highlighted  several  problems  faced  by  Wildland  Fire 
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Agencies  as  they  worked  to  incorporate  AAR  process.  Among  the  specific  problems  DeGrosky 
noted  were  applying  the  AAR  technique  without  regard  to  context,  using  AARs  inconsistently, 
practicing  informal  or  unstructured  AARs,  engaging  in  unsystematic  preparation  for  AARs,  and 
improperly  training  AAR  facilitators. 

A  third  issue  reflects  the  theoretical  grounding  of  the  technique.  AARs  are  meant  to 
serve  a  pedagogical  purpose:  to  enhance  learning  from  experience.  Their  structure  and  function 
should  follow  logically  from  theoretical  findings  in  various  scientific  domains.  These  include 
mainstream  psychology  (optimization  of  feedback,  training  methodology,  and  the  behavior  of 
small  groups  and  teams),  education  (curriculum  development  and  the  behavior  of  facilitators), 
instructional  technology  (development  and  incorporation  of  methods  for  data  gathering  and 
replay),  industrial  psychology  (design  of  performance  appraisal  systems,  leadership,  and  the  role 
of  organizational  influences),  knowledge  management  (organization  of  curricular  issues  and  their 
sequential  presentation  and  mastery)  and  military  planning  and  leadership.  Previous  attempts  to 
understand  the  AAR  process  have  typically  been  restricted  to  a  subset  of  these  areas,  or  have 
assumed  a  non-theoretically  based  approach.  Consideration  of  theory  will  avoid  duplication  of 
effort  and  ensure  that  the  AAR  represents  a  flexible  and  powerful  method  for  optimizing 
learning.  The  following  paragraphs  present  foundations  and  recommendations  for  AARs  that  are 
based  on  a  review  of  the  theoretical  literature. 

Theoretical  Background 

Learning  and  cognition.  Cognitive  learning  is  a  crucial  portion  of  the  AAR  process. 
Soldiers  must  reliably  observe  and  encode  trends  to  make  future  predictions  about  battle  status 
and  potential  outcomes.  Cognitive  learning  theory  concerns  transfer  of  information  from  short¬ 
term  to  long-term  memory. 

There  are  two  main  components  of  the  learning  process  as  it  pertains  to  the  long  term 
memory  storage  (LTS).  Encoding  is  the  process  in  which  information  gets  "stored"  into  long 
term  memory,  while  retrieval  deals  with  accessing  the  information  previously  stored.  When 
information  is  encoded  into  LTS,  it  is  usually  placed  into  a  "cognitive  network"  or  schema. 

When  presented  with  an  item  from  a  schema,  other  items  in  the  schema  are  subsequently  easier 
for  the  brain  to  access  (Crestani,  1997).  As  items  occur  together  repeatedly  in  a  person's 
environment,  the  association  between  them  becomes  stronger.  Decay  occurs  when  items  in 
memory  lose  their  connection,  which  occurs  when  there  are  long  spans  of  time  in  which  the 
items  are  not  experienced  or  paired  together  (Wickens,  Lee,  Liu,  &  Becker,  2004).  To  take 
advantage  of  this  cognitive  structuring,  AARs  should  focus  on  frequently  and  consistently 
providing  multiple  deep  and  meaningful  cues  to  important  items  in  memory. 

Episodic  memory  is  a  term  given  to  a  learner's  representation  of  an  event  and  its  relevant 
details.  Researchers  have  demonstrated  that  individual  encoding  is  susceptible  to  flaws, 
especially  under  stressful  conditions  (Wright  &  Davies,  2007).  Llaws  can  occur  due  to  limited 
attention  capacity  during  the  event,  or  the  mind  incorrectly  "filling  in  the  blanks"  of  what  was 
missed  perceptually  (Hyman,  1999).  AAR  practitioners  should  be  aware  of  problems  with  faulty 
episodic  memory,  because  acceptance  of  a  common  "ground  truth"  is  crucial  for  acceptance  of 
the  feedback  given  in  the  AAR. 

Situation  awareness  (SA),  another  cognitive  construct  of  interest,  is  defined  as  a  person's 
general  understanding  of  real  time  changes  in  their  surrounding  environment  (Ends ley,  1995). 
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There  are  three  levels  of  SA:  perceiving  elements  in  the  environment,  understanding  how  the 
elements  interact,  and  predicting  how  the  elements  will  interact  in  the  near  future  (Durso  & 
Gronlund,  1999).  AARs  should  allow  participants  to  improve  SA  by  highlighting  which  aspects 
of  the  battle  were  important  to  attend  and  which  aspects  were  irrelevant.  Facilitators  should  also 
be  concerned  with  increasing  the  SA  of  the  group  as  a  whole,  which  involves  facilitating  team 
members’  sharing  crucial  situational  information. 

The  cognitive  factors  just  listed  depend  greatly  on  individual  effort.  If  knowledge  is 
deemed  to  be  too  costly  compared  to  the  benefits,  learners  will  benefit  slowly  (if  at  all) 

(Sternberg  &  Smith,  1988).  If  information  is  deemed  trivial  or  unimportant,  little  or  no  effort 
will  be  made  to  attend  the  information  presented.  Facilitators  should  take  proactive  steps  to 
highlight  the  potential  value  of  understanding  a  process,  situation,  or  phenomenon  to  ensure  that 
important  aspects  of  the  lessons  are  retained. 

Human  cognition  can  also  be  limited  by  a  person's  attention  capacity.  Tasks  may  require 
more  attention  than  a  single  individual  is  able  to  give.  Given  this,  one  potential  focal  point  of  an 
AAR  is  to  identify  task  breakdowns  due  to  an  individual  becoming  overloaded,  and  discussing 
potential  ways  of  dividing  up  the  task  among  multiple  individuals. 

AARs  should  identify  those  individuals  who  are  experts  or  quickly  approaching  expertise 
and  encourage  these  individuals  to  participate  in  the  discussion.  Less  experienced  team  members 
can  gain  considerable  benefit  from  their  perspective  and  experience. 

Team  dynamics.  A  team  is  defined  as  a  small  number  of  people  who  hold  specialized 
roles  or  functions,  and  act  dynamically  towards  a  shared  goal.  The  individuals  of  the  team  are 
interdependent  and  seen  as  a  "working  unit".  Each  member  in  the  team  should  understand  the 
other  members’  roles  in  addition  to  their  own  (Belbin,  1981),  and  communication  is  crucial  for 
success.  Military  teams  include  Soldiers  with  differing  ranks  and  experience,  so  when  one 
member  of  a  team  is  incapacitated  another  member  may  need  to  assume  the  role.  Such  transfer  of 
duties  and  responsibilities  often  proves  to  be  a  considerable  challenge  for  members  who  have  not 
discussed  the  process  before  it  happens  or  who  may  not  have  adequately  acquired  or  refreshed 
their  knowledge  of  others’  roles.  AARs  are  an  appropriate  time  to  discuss  contingency  plans  if  a 
team  member  should  become  incapable  of  performing  their  duties.  In  addition,  the  discussion 
may  invoke  a  greater  appreciation  and  understanding  for  other  team  members’  roles,  which  in 
turn  could  impact  a  team’s  situation  awareness  and  cohesiveness. 

Communication  is  another  crucial  aspect  of  team  performance.  AARs  provide  the 
mechanism  to  identify  and  explore  any  communication  breakdowns  among  team  members. 

These  situations  should  be  explored  and  the  participants  should  be  encouraged  to  discuss  why  it 
happened  and  potential  fixes  to  avoid  a  breakdown  in  the  future.  AAR  facilitators  can  serve  to 
make  team  members  better  aware  of  communication  deficiencies  or  breakdowns,  and  may  be 
able  to  suggest  strategies  for  preventing  such  problems  and  keeping  the  team  focused  on  the  task. 

Goal  setting  activities  for  individuals  and  the  overall  team  have  been  demonstrated  to  be 
very  important  in  the  performance  appraisal  process  (DeShon,  Kozlowski,  Schmidt,  Milner,  & 
Wiechmann,  2004).  Having  a  specific  goal  to  realize  or  discuss  will  provide  a  concrete,  tangible 
item  to  discuss  in  an  AAR.  Any  references  made  by  the  trainees  to  performance  goals  should  be 
encouraged  and  fully  discussed. 
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Poor  team  performance  can  result  from  a  variety  of  causes.  Members  of  the  team  may 
disagree  over  the  authority  within  the  group,  causing  a  loss  of  identity  with  the  team  and  team's 
shared  goals.  Team  performance  can  break  down  when  individuals  manifest  incongruent  norms, 
low  morale,  poor  team  structure  or  roles,  deficiencies  in  communication,  and  lack  of  appropriate 
feedback  or  criticism.  Rewarding  or  punishing  a  specific  individual  can  lead  to  decreased  team 
goal  commitment;  for  that  reason  feedback  should  often  be  directed  at  the  group  as  a  whole  to 
ensure  team  accountability  and  cohesiveness  (Nadler,  1979). 

Feedback.  Feedback,  the  heart  of  the  AAR  process,  concerns  providing  individuals  or 
groups  with  information  about  their  performance  on  a  task  or  set  of  tasks.  Feedback  is  generally 
supported  in  being  an  important  factor  in  increasing  subsequent  performances  of  a  task  (London, 
2003).  Theoretically,  feedback  consists  of  a  source,  a  message,  and  a  recipient,  all  of  which  play 
special  roles  in  the  feedback  process. 

Aspects  of  feedback  are  available  from  observers  of  the  performance,  the  task 
environment  itself,  and  the  individual  learner.  Although  feedback  during  an  AAR  frequently 
comes  from  outside  observers,  it  is  important  to  consider  the  other  two  sources.  Task 
environment  feedback  and  self-feedback  are  often  well  accepted  and  incorporated  (Greller  & 
Herold,  1975);  however,  if  such  feedback  is  incongruent  with  the  overall  AAR,  learners  could 
possibly  disregard  information  within  the  AAR.  Facilitators,  when  possible,  should  try  to  make 
sure  that  all  sources  point  to  a  common  conclusion.  In  modem  training  situations  the  task 
environment  may  be  the  best  source  for  ground  truth  and  can  provide  the  best  feedback  on  what 
happened.  This  serves  to  help  eliminate  any  incongruity  among  or  between  observers  and 
individual  learners. 

Individuals  providing  feedback  may  differ  in  terms  of  their  credibility  and  power.  More 
credible  sources  will  have  their  feedback  accepted  and  acted  upon  more  readily,  whereas  sources 
that  are  deemed  non-credible  will  have  their  feedback  regarded  as  faulty,  incorrect,  or  irrelevant. 
Individuals  perceived  as  having  more  power  will  have  their  feedback  attended  to  regardless  of 
the  feelings  of  the  recipient.  Given  this  information,  a  facilitator  should  take  great  care  in 
establishing  and  maintaining  good  credibility,  and  ensuring  that  all  participants  feel  the  feedback 
given  refers  to  their  actual  performance. 

The  information  conveyed  within  an  AAR  should  be  clear  and  non-ambiguous,  as 
nonspecific  feedback  will  have  little  effect  on  performance  (Annet,  1969).  The  feedback  message 
should  reduce  uncertainty  in  terms  of  the  outcome  of  a  particular  task.  AARs  should  focus  on 
delivering  clear,  specific,  unambiguous  feedback  to  the  training  audience  that  reduces 
uncertainty  as  to  what  happened  and  sets  the  conditions  for  introspection  and  discussion  that  will 
enable  the  trainees  to  reach  a  clear  consensus  on  the  reasons  why  a  particular  outcome  occurred. 

Whereas  the  above  paragraphs  have  generally  involved  the  AAR  administrator’s 
perspective,  it  can  be  very  instructive  to  adopt  the  learner’s  perspective.  There  are  four  levels  of 
recipient's  processing  of  feedback:  perception  of  the  feedback,  acceptance  of  the  feedback,  desire 
to  respond  to  the  feedback,  and  the  intended  response  (Ilgen,  Fisher  &Taylor,  1979).  Each  of 
these  stages  can  be  impacted  by  the  three  component  sources:  the  source,  the  message,  and  the 
recipient. 

Perceptions  of  feedback  that  can  be  altered  by  the  source,  such  as  self  and  task  feedback, 
are  generally  perceived  more  readily  than  outside  observer  feedback.  Feedback  messages  can 
differ  in  timing,  sign,  and  frequency.  Timing  refers  to  the  amount  of  time  between  the  behavior 
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or  task  in  question  and  the  delivered  feedback.  A  closer  pairing  facilitates  the  linkage  between 
the  task  and  the  feedback  received.  The  sign  of  the  feedback  refers  to  positive  or  negative,  with 
negative  feedback  informing  the  individual  when  a  task  was  done  improperly  or  incorrectly. 
Negative  feedback  is  not  perceived  as  readily  as  positive  feedback  (Halperin,  Snyder,  Shenkel,  & 
Houston,  1976).  Finally,  the  frequency  of  the  feedback  is  how  often  it  is  delivered.  Frequent, 
consistent  feedback  will  have  more  of  an  effect  on  subsequent  performance. 

Acceptance  of  feedback  is  the  stage  in  which,  after  being  perceived,  information  is 
judged  to  be  accurate  or  not.  The  receiver  decides  whether  the  feedback  corresponds  to  displayed 
performance.  If  so,  feedback  will  likely  be  accepted.  Sources  deemed  more  credible  will  have 
their  feedback  accepted  more  readily,  and  positive  feedback  seems  easier  for  individuals  to 
accept  than  negative  feedback.  Desire  to  respond  is  the  stage  in  which  the  receiver  decides  if 
they  will  alter  their  task  performance  based  on  the  feedback  received.  Feedback  should  be 
designed  to  bolster  the  recipients’  feelings  of  competency,  while  reducing  the  feeling  of  external 
control.  Where  possible,  feedback  should  be  positively  framed,  and  should  include  information 
above  and  beyond  the  information  already  retained  by  the  recipient. 

The  final  stage  is  the  intended  response.  A  learner’s  intended  response  often  corresponds 
heavily  with  goal  setting  and  checking,  as  feedback  is  essential  to  the  goal  process.  Specific 
goals  lead  to  a  more  substantial  increase  in  performance  than  general  goals  that  fail  to  guide 
specific  behavior  (Steers  &  Porter,  1974).  Difficult  goals  also  stimulate  increased  performance 
more  than  easy  ones.  The  more  personal  control  over  goals  set,  the  more  performance  will 
increase.  To  support  specific  goals,  specific,  actionable  feedback  should  be  provided.  Self 
developed  actionable  feedback  will  be  accepted  by  all,  therefore  the  facilitators  have  the 
responsibility  to  use  appropriate  techniques  to  help  trainees  to  develop  that  consensus  in  order  to 
take  advantage  of  this. 

The  four-step  process  just  described  suggests  that  AAR  facilitators  should  attempt  to 
ensure  that  each  person  in  the  AAR  agrees  with  the  feedback  that  is  given  so  that  it  will  not  be 
disregarded.  Feedback  should  also  be  delivered  in  a  timely  fashion.  Because  self-feedback  will 
be  accepted  more  readily,  facilitators  should  try  to  have  participants  generate  their  own  personal 
feedback  when  addressing  issues.  AAR  facilitators  should  work  hard  to  maintain  themselves  as 
credible,  energetic,  reliable,  and  representing  the  participants’  best  interests.  The  AAR  designer 
and  AAR  facilitator  should  also  help  the  participants  develop  challenging,  specific  goals  for 
future  performance,  and  subsequently  offer  specific  feedback  to  provide  a  means  of  monitoring 
progress  towards  these  goals. 

As  our  research  team  became  familiar  with  the  applied  military  practice  of  AARs,  we 
evaluated  our  observations  against  the  preceding  theoretical  conclusions.  In  turn,  we  refined  our 
theoretical  understanding  of  AAR  construct  foundations  by  considering  real-world  constraints. 

Military  Research  Efforts 

We  identified  in  available  literature  key  individual  researchers  in  the  military  domain 
who  have  contributed  to  understanding  and  improving  the  AAR  process.  Among  these  are  Larry 
Meliza,  David  Bessemer,  and  John  Morrison.  An  early  report  concerning  the  implementation  of 
AAR  within  the  SIMNET  architecture  was  authored  by  Meliza  along  with  personnel  from  the 
Fort  Knox  field  unit  of  the  Army  Research  Institute  (Shlechter,  Meliza,  Burnside,  &  Bessemer, 
1992).  Other  works  by  Meliza  have  included  a  1999  report  addressing  the  automation  of  AARs 
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(Meliza,  1998)  as  well  as  a  landmark  publication  with  Morrison  concerning  the  foundations  of 
the  AAR  method  (Morrison  &  Meliza,  1999). 

The  agencies  or  organizations  that  have  concerned  themselves  with  AARs  span 
government  agencies  as  well  as  industry.  Each  of  the  armed  forces  has  adopted  a  version  of 
AAR,  with  the  U.S.  Army  leading  the  way.  The  most  influential  military  research  concerning 
AAR  has  been  conducted  by  the  U.S.  Army  Research  Institute  and  the  Center  for  Army  Lessons 
Learned  (CALL).  These  groups  have  produced  numerous  reports  and  guides  that  describe 
methods  for  conducting  AARs  and  often  include  recommendations  for  their  effective 
implementation  and  facilitation.  Most  of  the  military  AAR  research  in  the  past  10  years  has 
shifted  from  foundational  and  theoretical  work  to  application  of  the  techniques  under  emerging 
training  technology  conditions  or  development  of  technologies  to  support  the  established  AAR 
process.  The  fundamental  research  conducted  prior  to  1999  has  been  well  documented  and 
described  in  other  works  (Salter  &  Klein,  2007;  Mastaglio,  Jones,  Bliss  &  Newlin,  2007)  and 
will  not  be  repeated  here.  Below  is  a  discussion  of  some  of  the  significant  efforts  conducted  by 
military  researchers  in  the  more  recent  past  to  extend  AAR  into  emerging  application  areas  or  to 
exploit  emerging  technologies  to  enhance  the  conduct  of  the  AAR. 

The  Army’s  Functional  Area  (FA)  57,  Simulation  Operations,  Proponency  Office 
published  an  extensive  DVD-based  AAR  Toolkit  that  provides  multi-media  instruction, 
recommendations  and  a  comprehensive  set  of  example  templates  for  trainers  to  use  in  the 
preparation  and  facilitation  of  AARs  (FA-57  Proponency,  2003).  This  DVD  was  based  on  the 
conduct  of  planning,  preparing,  rehearsing,  and  conducting  AARs  as  executed  at  the  Combat 
Training  Centers  (CTCs).  Though  it  is  an  extensive  resource,  most  home  station  trainers  do  not 
have  the  infrastructure  available  to  implement  AARs  as  described  in  the  DVD.  This  toolkit  has 
not  been  widely  distributed  beyond  FA  57  personnel  so  it  had  relatively  little  impact  on  the  large 
population  of  Army  trainers. 

As  part  of  a  research  program  intended  to  enhance  data  collection  and  feedback 
capabilities  in  virtual  training  environments  for  dismounted  Soldiers,  ARI  developed  the 
Dismounted  Infantry  Virtual  After  Action  Review  System  (DIVAARS)  (Knerr,  Lampton, 

Martin,  Washburn,  and  Cope,  2002).  DIVAARS  provides  a  comprehensive  suite  of  tools  and 
capabilities  to  address  the  unique  challenges  and  advantages  of  conducting  AARs  of  virtual 
training  exercises  intended  to  support  training  of  dismounted  Soldiers.  The  system  was 
envisioned  to  work  with  any  Distributed  Interactive  Simulation  (DIS)  compliant  virtual 
environment.  DIVAARS  received  very  positive  ratings  from  Soldiers  who  participated  in  AARs 
conducted  during  the  development  of  the  system  and  it  has  served  as  a  capabilities  benchmark 
for  follow-on  efforts  by  both  commercial  and  government  organizations. 

Another  prominent  researcher  who  contributed  to  the  potential  for  applying  emerging 
technologies  to  the  enhancement  of  AARs  is  Jean  Dyer  (Dyer,  Wampler,  Blankenbeckler,  2005). 
Dyer  et  al.  explored  the  potential  to  leverage  the  operational  capabilities  (computing  power  and 
information  available)  of  the  Land  Warrior  system  to  provide  AAR  aids.  The  Land  Warrior 
system  is  a  future  dismounted  Soldier  system  that  includes  a  wearable  computer  and  helmet- 
mounted  display.  This  research  addressed  how  such  a  Soldier  system  could  be  used  to  provide 
AAR  aids  to  enhance  the  trainer's  AAR  dialogue  with  the  unit,  as  well  as  what  additional  training 
capabilities  could  be  embedded  in  the  system's  design  to  support  the  AAR. 
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Other  researchers  also  explored  the  potential  opportunities  and  challenges  associated  with 
the  emerging  ubiquitous  employment  of  computer-based  information  systems.  Meliza  and 
Barnett  (2006)  researched  the  extent  to  which  U.S.  Army  units  equipped  with  emerging 
networked  command  and  control  systems  must  channel,  manage,  assess,  and  exploit  information 
and  requests.  They  observed  that  the  building  block  relationships  among  these  skills  (e.g., 
channeling  facilitates  management)  relates  directly  to  the  crawl-walk-run  approach  to  training. 

At  lower  levels  of  proficiency  in  applying  networked  systems,  more  effort  should  be  spent  on 
information  channeling  and  management,  while  assessment  and  exploitation  become  the  foci  at 
higher  levels  of  proficiency.  Their  assessment  of  the  requirement  for  the  AAR  process  whereby 
organizations  decide  what  happened,  why  it  happened,  and  how  to  improve  or  sustain  future 
performance  concluded  that  it  is  “likely  to  be  a  key  feedback  mechanism  for  training  units  using 
networked  systems”.  Just  as  Dyer,  et  al,  identified  the  opportunity  to  exploit  the  operational 
information  provided  by  these  systems  as  sources  of  AAR  aids,  Meliza  and  Barnett  concluded 
that  the  information  available  from  the  networked  systems  could  provide  information  relevant  to 
identifying  problem  areas.  This  paper  also  addresses  how  the  levels  of  operator  network 
proficiency  can  affect  the  need  for  AAR  aids. 

Military  research  to  develop  tools  to  support  new  training  conditions  includes  efforts  such 
as  the  After  Action  Intelligent  Review  System  (AAIRS)  application  developed  for  the  Marine 
Corps  Combined  Arms  Command  and  Control  Trainer  Upgrade  System  (CACCTUS)  program 
(Jensen,  Harmon,  Nolan  &  Caldwell,  2006).  This  effort  is  focused  on  a  visual  timeline -based 
debrief  toolset  that  enables  instructors  to  quickly  construct  and  present  playback  vignettes  for 
salient  training  points.  It  identifies  training  points  through  the  use  of  intelligent  data  collection 
and  causal  analysis  methods.  AAIRS  records  the  training  mission  execution  that  includes  a 
synchronized  collection  of  exercising  force  audio  communications  and  human  in  the  loop 
interaction  with  system  components  (operational  C4I  and  simulator  interface  tools).  Key  exercise 
data  relevant  to  training  points  is  tabulated  by  the  debrief  construction  tool  and  presented 
visually  along  a  timeline  for  instructor  review.  This  facilitates  a  highly  configurable  AAR,  where 
the  presentation  of  each  training  point  in  the  debrief  can  include  descriptive  causal  analysis  text, 
voice  communications  playback,  and  3D  views  of  the  battlefield,  either  frozen  in  time  (visually 
as  a  snapshot)  or  played  back  as  vignettes. 

Salter  and  Klein’s  (2007)  research  report  is  highly  relevant  and  timely  for  this  research. 

In  cooperation  with  the  Operations  Group  of  the  Joint  Readiness  Training  Center  (JRTC)  at  Fort 
Polk,  LA,  they  sought  to  provide  an  independent  assessment  of  the  conduct  of  AARs  at  JRTC. 
The  JRTC  is  a  key  venue  for  AAR  analysis  because  the  likelihood  of  small  unit  leader 
involvement  in  contemporary  decision-making  situations  has  increased.  The  greater  participation 
of  junior  leaders  and  Soldiers  has  brought  a  renewed  focus  on  AARs  conducted  at  platoon  and 
company  level.  The  researchers  observed  forty  small  unit  (platoon  and  company)  JRTC  AARs  in 
real  time  and  via  videotape.  The  AARs  were  compared  to  each  other,  and  most  importantly,  to 
the  Army  standard  as  shown  in  Training  Circular  25-20,  A  Leader's  Guide  to  After- Action 
Reviews  (Department  of  the  Army,  1993).  They  also  interviewed  current  and  former  O/Cs,  and 
observed  instruction  provided  to  new  O/Cs  to  supplement  these  observations.  Current  findings 
were  compared  with  results  of  other  research  on  AARs  and  similarities  noted.  As  a  deliberate 
byproduct  of  the  research,  a  prototype  AAR  rating  scale  was  created  for  possible  use  as  a  job  aid 
or  performance  checklist  or  for  use  as  an  instructional  tool  during  O/C  training.  The  results  of 
this  research  confirm  earlier  findings  indicating  that  the  AAR  must  be  both  a  science  and  a  craft. 
The  steps  in  an  AAR  are  well  detailed  in  multiple  source  materials,  and  most  O/Cs  observed 
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appeared  to  understand  the  basic  requirements.  The  O/Cs  were  well-trained,  enthusiastic,  and 
clearly  interested  in  helping  the  units.  However,  many  of  even  the  most  proficient  O/Cs  tended  to 
err  on  the  side  of  providing  too  much  information,  thereby  turning  the  AAR  into  a  critique  or 
lecture  instead  of  a  discussion. 

Another  area  of  recent  interest  to  military  researchers  is  exploration  of  emerging 
technology  that  enables  computer  generated  training  performance  feedback  mechanisms 
intended  to  replace  the  human  trainer  or  AAR  facilitator.  Scott  Beal  (2008)  explored  the 
application  of  a  game-based  training  system  both  with  the  support  of  an  instructor  facilitator  and 
as  a  standalone  tactical  training  game.  In  this  effort  sixty-nine  Infantry  leaders  participated  in  an 
experiment  observing  the  difference  between  instructor-facilitated  vs.  stand-alone  game -based 
training.  The  instructor-facilitated  group  worked  with  instructors  and  peers  during  their  training 
exercises,  while  the  stand-alone  group  performed  the  training  missions  with  no  interaction  with 
others.  A  questionnaire  was  administered  post-exercise  to  assess  the  participant's  perception  of 
training  value,  tactical  decision  implementation,  and  motivation.  Results  indicated  that  there  was 
no  difference  in  the  decision  making  performance,  however  the  leaders'  perceived  training  value 
and  decision  making  was  impacted  positively  when  training  was  accompanied  by  peers  and/or 
trainers. 

There  has  been  a  re-emergence  of  research  efforts  to  explore  improving  the  efficacy  of 
more  modem  approaches  to  Intelligent  Tutoring  Systems  (ITSs)  to  better  support  performance 
feedback  using  reflective  tutoring  in  lieu  of  trainer  facilitated  AARs.  One  can  readily  find 
successful  implementations  of  ITSs  in  support  of  the  training  of  well  defined  tasks.  However, 
Chad  Lane’s  (Lane,  Core,  Gomboc,  Karnavat  &  Rosenberg,  2007)  work  addresses  the  much 
harder  problem  of  building  an  intelligent  tutoring  system  for  the  ill-defined  domain  of 
interpersonal  and  intercultural  skill  acquisition.  In  this  research  the  team  considers  mixed-result 
actions  (those  with  pros  and  cons),  categories  of  actions  (e.g.,  required  steps  vs.  rules  of  thumb), 
the  role  of  narrative,  and  reflective  tutoring,  among  other  topics.  The  context  for  this  work  is  an 
intelligent  tutor  for  ELECT  BiLAT,  a  game -based  system  to  teach  cultural  awareness  and 
negotiation  skills  for  bilateral  engagements.  The  tutor  provides  guidance  in  two  forms:  (1)  as  a 
coach  that  gives  hints  and  feedback  during  an  engagement  with  a  virtual  character  and  (2)  during 
a  system-generated  AAR  to  help  the  learner  reflect  on  choices  made.  Learner  activities  are 
mapped  to  learning  objectives,  which  include  whether  the  actions  represent  positive  or  negative 
evidence  of  learning.  These  underlie  an  expert  model,  student  model,  and  models  of  coaching 
and  reflective  tutoring  that  support  the  learner.  Durlach,  Wansbury,  and  Wilkinson  (2008) 
conducted  training  effectiveness  analyses  of  the  ELECT  BiLAT  system  as  a  standalone  training 
environment  with  its  ITS-based  AAR  system  and  demonstrated  a  significant  increase,  by  the 
Soldier  participants,  in  post  training  performance. 

The  above  research  focuses  primarily  on  technology  to  support  AARs.  To  date,  there  has 
been  little  research  to  identify  factors  that  support  or  inhibit  the  training  effectiveness  of  AARs. 
The  primary  motivator  of  the  current  research  was  the  lack  of  data  on  AAR  effectiveness  and  the 
factors  that  influence  that  effectiveness.  The  present  research  hopes  to  initiate  an  effort  to 
improve  the  training  effectiveness  of  AARs  in  the  US  Army. 

There  is  little  doubt  that  the  U.S.  military,  led  by  the  Army,  has  been  the  primary 
intellectual  energy  behind  the  concept  of  the  AAR.  The  above  is  only  a  sampling  of  research 
across  the  services  in  how  they  should  be  conducted  and  the  implications  for  technology-based 
solutions  for  training  and  support  of  the  AAR  process  itself.  In  a  similar  fashion,  the  military  has 
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developed  fundamental  doctrine  for  employing  the  AAR.  In  the  next  section  we  will  summarize 
the  written  guidance  developed  by  the  U.S.  Army  regarding  AAR. 

AAR  Doctrine 

The  lead  agency  for  both  training  and  doctrine  within  the  Army  is  the  U.S.  Army 
Training  and  Doctrine  Command  (TRADOC).  Therefore,  although  the  fundamental  Army 
research  agency  on  AAR  has  been  U.S.  Army  Research  Institute  for  Behavioral  and  Social 
Sciences,  the  command  responsible  for  directing  AAR  implementation  in  the  form  of  doctrine  is 
TRADOC.  TRADOC  Regulation  25-36,  The  TRADOC  Doctrinal  Literature  Program  (2004) 
states  that  the  purpose  of  Army  doctrine  is. . . 

...standardize  military  principles,  terms,  and  TTP  throughout  the  U.S.  Army. 

They  form  the  basis  for  training,  and  the  training  products  that  support  it. 

Training  standards  provide  performance  baselines  to  evaluate  how  well  a  task  is 

executed.  Together,  doctrine  and  training  form  the  key  to  Army  readiness. 

The  regulation  additionally  states  that  the  Army’s  “expertise”  is  “founded,  to  a  large 
extent,  on  the  intellectual  capital  of  those  who  develop  and  contribute  to. .  .doctrine.”  The 
preparation  and  publication  of  doctrine  serves  as  the  Combination  function  within  the 
Socialization,  Extemalization,  Combination,  and  Internalization  (SECI)  model  of  the  creation  of 
organizational  knowledge  (Nonaka  &  Takeuchi,  1995).  Doctrine  collects  the  extracted 
expressed  and  analyzed  learning  of  individuals  within  the  organization  and  subsequently 
internalizes  that  learning  across  the  organization.  Studies  and  other  research  are  not  doctrine  but 
rather  the  extemalization  of  individual  knowledge — the  elicitation  of  otherwise  tacit  knowledge 
that  will  eventually  evolve  into  doctrine.  The  biggest  discriminator  between  research  and 
doctrine  is  that  research  is  likely  neither  read  nor  known  of  by  the  body  of  practitioners,  who  will 
be  influenced  by  doctrine. 

AAR  has  become  an  accepted  practice  across  the  Army  and  in  turn  the  AAR  concept  has 
become  imbedded  into  Army  doctrine.  Field  Manual  25-4,  How  to  Conduct  Training  Exercises 
(1984) — which  is  still  available  from  Army  Knowledge  On-Line  (AKO) — has  an  appendix  on 
post-exercise  activities,  which  describes  conducting  AARs.  Training  Circular  25-20,  A  Leader’s 
Guide  to  After  Action  Reviews  (1993)  is  the  Army’s  stand-alone  document  describing  AARs 
and  provides  guidance  on  their  conduct.  Field  Manual  7-1,  Battle  Focused  Training  (2003)  has 
an  appendix  on  the  AAR,  which  mirrors  the  information  found  in  TC  25-20.  The  Research  team 
found  additional  AAR  doctrine  within  an  appendix  in  FM  6-01.1,  Knowledge  Management 
Section  (2008).  Also,  the  Team  examined  MCRP  3-0A,  Unit  Training  Management  Guide 
(1996).  As  the  Team  was  interested  in  instructions  to  AAR  practitioners,  the  Team  reviewed 
each  manual  to  extract  specified  instructions.  The  Team  organized  these  extracted  instructions 
under  associated  study  Issues  and  Sub-Issues  and  then  conducted  analysis  to  identify  the  current 
state  of  AAR  doctrine. 

These  doctrinal  examples  provide  a  dated  perspective  of  the  AAR.  The  Army’s  objective 
is  to  update,  or  at  least  review,  doctrine  every  five  years  (TRADOC  Regulation  25-36,  2004). 
Army  doctrine  on  the  execution  of  the  AAR  reflects,  with  some  minor  updates,  the  doctrine 
contained  in  the  original  TC  25-20,  which  at  the  time  of  this  research  was  seventeen  years  old 
and  was  written  prior  to  significant  changes  in  both  training  modalities  and  in  training 
perspectives  and  approaches. 


10 


The  doctrinal  examples  describe  an  AAR  as  a  discrete  event  that  exists  almost  external  to 
the  training  process.  Guidance  on  the  integration  of  AARs  into  training  events  is  minimal  and 
very  general,  referring  to  scheduling  AARs  after  “critical”  events  without  defining  what 
constitutes  criticality.  Some  of  the  doctrine  provides  examples  of  critical  events,  examples  that 
look  to  be  based  upon  steps  within  general  tactical  sequences. 

Doctrine  briefly  describes  the  requirements  for  O/C  or  facilitator  selection,  specifically 
knowledge  and  rank,  with  knowledge  having  the  higher  priority.  Doctrine  also  provides  some 
guidance  to  O/Cs  on  pre-AAR  requirements,  specifically  on  the  necessity  for  accurate  and 
thorough  observation  capture  (note  taking)  and  positioning  for  effective  observation.  The 
doctrine,  however,  describes  an  O/C  as  a  discrete  actor  in  the  training  process,  separate  from  the 
training  audience  and  reporting  information  to  a  senior  O/C.  Doctrine,  which  is  heavily  based  on 
TC  25-20,  published  in  1993,  does  not  take  into  consideration  the  numerous  technological 
advances  that  today  afford  AAR  practitioners  a  more  complex  information  environment. 

Doctrine  fails  to  identify  linkages  between  the  AAR  and  other  aspects  of  the  training  or 
learning  process.  This  is  most  pronounced  in  its  failing  to  identify  linkage  between  the  AAR  and 
normal  facilitator-training  audience  interactions  including  one-on-one  reviews,  with  the 
impression  from  the  doctrine  that  the  AAR  is  the  first  and  only  interaction  between  the  facilitator 
and  the  training  audience. 

AAR  doctrine  becomes  much  more  prescriptive  when  discussing  the  execution  of  the 
actual  AAR,  providing  very  specific  instruction  on  the  sequencing  of  events  within  the  AAR. 

The  doctrinal  examples  describe  a  somewhat  directive  AAR,  implying  almost  complete  control 
by  the  facilitator  of  the  AAR  flow,  including  discussions.  This  includes  assumptions  that  the 
AAR’s  aim  is  to  include  all  observed  learning  points.  Execution  emphasizes  several  interim 
means  to  success,  especially  group  discussion,  but  is  very  sparse  on  instruction  to  facilitators  on 
how  to  achieve  these. 

The  primary  facilitator  behavior  discussed  is  the  use  of  open-ended  questions,  providing 
a  definition  for  the  reader  but  little  discussion  of  what  comprises  an  effective  open-ended 
question  or  how  to  best  deploy  them. 

The  doctrinal  examples  are  relatively  silent  on  what  occurs  after  an  AAR  from  the 
facilitator’s  perspective.  It  is  in  this  area  where  the  doctrine  mostly  portrays  the  AAR  as  a 
singular  event.  Post- AAR  instructions  are  directed  to  unit  commanders  vice  facilitators  and 
discuss  what  to  do  with  AAR  results,  specifically  retraining.  There  is  little  to  no  discussion  of 
what  the  facilitator  does  after  the  AAR,  especially  on  how  multiple  AARs  can  be  linked  to  cause 
the  most  effective  and  efficient  training. 

Method 


Observed  Practice 

A  major  effort  of  this  investigation  was  to  determine  the  current  state  of  AAR  practice 
across  varied  training  modalities  and  then  to  compare  that  state  to  the  theoretical  foundations  for 
AAR  as  discussed  above.  The  team  developed  an  understanding  of  the  current  state  of  the  AAR 
through  interviewing  AAR  practitioners  at  two  centers  of  Army  training,  the  NTC  and  the  BCTP, 
and  observing  AARs  as  practiced  by  these  organizations.  Additionally,  the  team  collected  data 
through  a  web-based  survey.  The  population  for  this  web-based  survey  was  defined  as  active 
duty  Soldiers  who  have  participated  in  one  or  more  AARs.  We  collected  sample  data  using 
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voluntary  responses  to  an  announcement  of  the  survey  deployed  on  Army  Knowledge  Online 
(AKO). 

The  research  team  used  an  established  methodology  for  eliciting,  consolidating, 
analyzing,  and  visualizing  individual  tacit  information.  This  methodology,  called  the  Study  of 
Organizational  Opinion  (S02)  (Mastaglio  &  Jones,  2009),  was  developed  from  Customer 
Relations  Management  (CRM)  principles,  and  has  been  proven  in  a  variety  of  research  efforts, 
including  assessment  of  training  and  elicitation  of  organizational  requirements.  S02  is  highly 
effective  as  an  interview  and  surveying  technique  to  acquire  the  in-depth  insight  required  for 
analyzing  organizational  change  or  transition.  The  research  team  chose  to  use  S02  for  this 
research  due  to  its  demonstrated  ability  to  quickly  and  accurately  extract  complex  insights  from 
participants  responses  and  inputs. 

S02  starts  by  mapping  the  organization  and  the  potential  survey  population.  The  next 
step  in  S02  is  to  develop  Issues  from  research  goals  and  objectives.  An  Issue  is  a  broad  inquiry, 
often  associated  with  executive  level  information  requirements.  Practitioners  then  conduct  a 
hierarchical  deconstruction  of  these  Issues  into  Sub-Issues  and  eventually  Questions.  Sub-Issues 
are  more  focused  inquiries  often  associated  with  management  level  information  requirements. 
Questions  are  crafted  so  they  can  be  delivered  to  and  answered  by  a  respondent  who  we 
reasonably  expect  to  have  the  experience  and  insight  to  provide  the  desired  information. 
Respondents  are  identified  concurrently  with  Issue  deconstruction,  and  are  logically  grouped 
based  upon  organizational  and  research  requirements.  Questions  are  deployed  to  respondents 
through  on-line  questionnaires,  written  surveys  or  interviews.  Answers  are  then  consolidated 
and  are  aggregated  into  Sub-Issue  Findings  and  eventually  Issue  Findings,  with  analysis  and 
sense-making  at  each  aggregation. 

This  investigation  started  with  the  hypothesis  that  the  AAR  has  become  a  significant 
component  of  training  and  therefore  there  are  key  implications  that  must  be  considered 
throughout  the  training  management  cycle.  To  provide  structure  to  this  investigation,  the  team 
started  with  an  existing  training  management  model.  The  team  evaluated  two  training 
management  models,  the  US  Army’s  Training  Management  Cycle  and  the  US  Joint  Forces 
Command’s  (JFCOM)  Joint  Training  System  (JTS). 

The  team  initially  examined  the  U.S.  Army’s  Training  Management  Cycle  (FM  7-1, 
2003).  This  training  model  consists  of  four  sequential  processes:  Mission  Essential  Task  List 
(METL)  Development,  Planning,  Execution,  and  Recovery.  The  process  we  focused  on  for  this 
investigation  was  Execution.  Execution  is  divided  into  three  components:  Preparation  for 
Training,  Conduct  of  Training,  and  Recovery  from  Training.  According  to  FM  7-1,  AARs  occur 
during  the  Conduct  of  Training  component. 

The  JTS  consists  of  four  phases  that  are  similar  to  the  Army  model:  Requirements, 

Plans,  Execution,  and  Assessment  (CJCSM  3500.03 A,  2002).  As  with  the  Army’s  model,  this 
research  focused  on  the  third  phase  of  the  JTS,  Execution,  during  which  the  AAR  occurs.  The 
Execution  phase  of  JTS  is  described  by  the  Joint  Exercise  Life  Cycle  (JELC).  The  JELC  has  five 
stages:  Design,  Plan,  Prepare,  Execute,  and  Evaluate.  The  AAR  is  included  in  the  Execute 
stage.  After  comparing  the  two  models,  we  chose  the  JTS/JELC  because  it  provides  more 
structure  and  more  detailed  definition  of  the  training  management  process  than  the  Army  model. 
The  team  based  the  research  Issues  on  the  stages  of  the  JELC  combining  the  first  three  stages, 
Design,  Plan,  and  Prepare,  into  one  Issue.  The  key  Issues  for  this  effort  were: 
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Issue  1:  How  are  AARs  considered  during  the  Design,  Planning,  and  Preparation  of  Training? 
Issue  2:  How  are  AARs  considered  during  the  Execution  of  Training? 

Issue  3:  How  are  AARs  considered  during  the  Evaluation  of  Training? 

NTC  and  BCTP  Interview  Methodology 

For  the  AAR  practitioner  interview  portion  of  the  investigation,  each  of  these  Issues  were 
deconstructed  into  nine  Sub-Issues  and  forty-nine  interview  Questions.  The  AAR  occurs  in  the 
Execution  Phase,  therefore,  most  emphasis  was  placed  on  Issue  2.  In  addition  to  Questions 
developed  from  Issues,  there  were  eleven  demographic  Questions  and  four  additional  Questions 
that  supported  this  effort  but  did  not  derive  directly  from  one  of  the  three  above  issues,  for  a  total 
of  sixty-four  questions. 

Questions  were  presented  to  respondents  during  face-to-face  interviews  conducted  at  the 
NTC  over  a  three-day  period  and  at  BCTP  over  a  four  day  period.  With  the  exception  of  several 
demographic  questions,  all  questions  were  open-ended  and  the  interviewer  was  allowed  to 
follow-up  to  refine  respondent  answers.  Thus,  each  interview  was  extensive,  lasting  on  average 
two  hours.  Prior  to  interviews  at  the  NTC,  a  rehearsal  was  conducted  at  the  U.S.  Army’s  JRTC. 
This  rehearsal  served  to  select  and  refine  questions  and  improve  interview  techniques.  In  all,  we 
observed  five  battalion  and  one  brigade  AAR  at  NTC,  and  one  AAR  at  BCTP.  We  interviewed 
six  OCs  at  NTC  and  four  at  BCTP.  In  addition,  at  JRTC  we  observed  one  company  and  one 
brigade  AAR,  and  interviewed  three  OC  teams  of  one  officer  an  done  NCO  (total  of  six  OCs). 

Upon  returning  from  the  NTC  and  BCTP,  we  archived  the  twenty-plus  hours  of  interview 
protocols  into  a  central  database.  An  analysis  of  individual  interviewee  responses  followed. 
Following  the  S02  methodology,  these  were  aggregated  into  Consensus  Answers  and  Sub-Issue 
Findings  for  each  organization  (NTC  and  BCTP)  separately.  These  Sub-Issue  Findings  by 
organization  were  then  consolidated  into  an  overall  Consensus  Sub-Issue  Findings  and  Issue 
Findings.  Organizational  and  Consensus  Sub-Issue  Findings  and  Issue  Findings  are  provided 
below. 

Task  Analysis  Data  Gathering  Process 

A  central  part  of  our  experimental  efforts  was  to  observe  the  practice  of  AARs  in  U.S. 
Army  units.  To  accomplish  this  goal,  two  forms  of  a  cognitive  task  analysis  questionnaire  were 
developed  according  to  the  procedures  outlined  by  Crandall,  Klein,  &  Hoffman  (2006).  In  ideal 
task  performance  situations,  task  analysts  invest  several  hours  to  observe  and  interview  task 
performers.  They  also  engage  subject  matter  experts  in  the  construction  and  modification  of 
AAR  observational  tools.  However,  time  and  access  constraints  prohibited  us  from  adhering  to 
an  exhaustive  task  analysis  process.  We  created  a  structured  observation  process  that  would 
allow  our  team  to  draw  accurate  and  detailed  conclusions  about  the  practice  of  AARs,  one  that 
would  inform  existing  theory  in  the  areas  described  earlier.  In  so  doing,  we  attempted  to  stay  as 
true  as  possible  to  accepted  task  analysis  practice.  Our  efforts  were  based  upon  our  previously 
developed  theory  which  we  were  seeking  to  validate,  the  ITAAR  developed  by  Mastaglio,  Jones, 
Newlin,  and  Bliss  (Mastaglio  et  al.,  2007).  The  ITAAR  model  is  shown  in  Figure  1  (for  a 
thorough  description  of  the  ITAAR  model,  see  Bliss,  Minnis,  Wilkinson,  Mastaglio  &  Barnett,  in 
preparation). 
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-  Knowledge  of  task  facilitates  goal  setting;  goals  structure  the  knowledge 
dispersement  process. (9) 

-  Feedback  is  formalized  within  performance  appraisals;  performance  appraisals 
are  the  mechanism  for  feedback. (1 0) 


1  Wiener  (1948) 

2  Saavedra ,  Earley  &  Van  Dyne  (1993);  Matsui,  Kakuyama  &  Onglatco  (1987) 

3  Mager  (1984) 

4  Cannon-Bowers  &  Salas  (2001) 

5  Smith  (2001) 

6  Schmidt  &  Kleinbeck  (1990) 

7  Jones  (1997) 

8  Dyer  (1986) 

9  Dyer  ( 1 986);  Saavedra  et  al.  ( 1 993) 

10  Patrick  (1992) 


Figure  1.  The  integrated  theory  of  after  action  review  (Mastaglio  et  al.,  2007). 
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To  facilitate  our  data  gathering,  one  questionnaire  included  items  to  be  answered  by 
military  subject  matter  experts  familiar  with  the  AAR  process.  The  other  form  of  the 
questionnaire  consisted  of  a  checklist  of  items  drawn  from  research  literature  concerning 
effective  AARs  (Mastaglio,  Jones,  Newlin  &  Bliss,  2007).  The  items  on  the  questionnaire  and 
the  checklist  reflected  the  theoretical  structure  of  the  IT AAR.  The  observations  that  we  made 
represented  frequencies  observed  for  each  item  on  the  checklist. 

The  specific  behavioral  items  included  on  the  checklist  and  the  IT  AAR  construct  they 
relate  to  were  as  follows: 

•  Gives  Individual  Praise  (feedback) 

•  Gives  Individual  Criticism  (feedback) 

•  Gives  Group  Praise  (feedback) 

•  Gives  Group  Criticism  (feedback) 

•  Asks  a  Rhetorical  Question  (learning) 

•  Asks  a  Question  Directed  at  the  Group  (team  operations) 

•  Asks  a  Question  Directed  at  an  Individual  (learning) 

•  Encourages  Group  Discussion  (team  operations) 

•  Engages  in  Lecturing  or  Instructing  (leadership) 

•  Refers  to  a  Performance  Goal  (learning) 

•  Discusses  an  Action  Summary  (feedback) 

•  Uses  a  Technical  Aid  (leadership) 

•  Uses  an  Example  to  Clarify  a  Point  (leadership) 


Following  development  of  the  checklist,  the  experimenters  arranged  to  observe  six  2-hour 
AARs  at  the  NTC,  Ft.  Irwin,  CA,  and  additional  AAR  sessions  at  Ft.  Stewart,  GA,  Ft.  Polk,  LA, 
and  Ft.  Leavenworth,  KS.  AARs  observed  were  at  the  brigade  and  company  level.  All 
observations  were  conducted  in  real  time  as  the  events  unfolded.  However,  at  the  request  of  the 
government,  the  research  team  minimized  its  obtrusiveness  by  observing  many  of  the  AAR 
sessions  at  a  remote  facility  by  video  camera. 

Web-Based  Survey  Methodology 

Our  interviews  of  AAR  facilitators,  as  discussed  above,  provided  their  perspective  on  the 
AAR,  however  these  interviews  did  not  include  perspectives  from  the  other  group  of  major  AAR 
participants,  the  training  audience.  To  obtain  this  perspective,  the  team  created  an  online  web 
survey.  Following  MYMIC’s  S02  process,  the  team  crafted  questions  based  upon  existing 
study-matrix  Issues  and  Sub-Issues.  The  team  decided  that  Issue  3:  “How  are  AAR  results 
incorporated  in  future  AAR  practices?”  was  not  pertinent  to  our  survey  of  training  audience 
participants  and,  therefore,  no  questions  addressed  this  issue.  The  resulting  on-line  questionnaire 
contained  twenty-seven  questions. 

In  order  to  maximize  survey  completion,  we  crafted  mostly  closed  questions — those  with 
either  multiple  choice  or  Likert  scale  answers.  Within  the  survey,  there  were  six  sets  of 
questions  grouped  by  common  theme  (i.e.,  sets  of  Likert  score  questions  regarding  the  use  of 
audio,  video,  etc.  within  the  AAR  were  grouped  together).  These  sets  each  provided  insight  at 
both  the  high  level,  in  terms  of  information  regarding  the  overarching  question,  and  at  the 
individual  question  level.  In  order  to  maximize  the  opportunity  for  additional  feedback,  there 
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were  also  three  open-ended  questions  at  the  end  of  the  survey  to  allow  respondents  to  provide 
unconstrained  responses. 

Table  2.  Breakdown  of  Questions  by  Issue 


Issue 

#of 

Questions 

Demographics 

6 

Issue  1 

6 

Issue  2 

12 

Issue  3 

0 

Other 

3 

Total 

24 

The  survey  collected  limited  demographic  information.  Due  to  the  breadth  of  the  survey 
population — serving  Army  personnel — the  survey  team  decided  not  to  pre-identify  respondent 
groups. 

The  team  deployed  the  survey  via  an  announcement  on  the  AKO  main  page.  For  the 
convenience  of  users,  the  team  built  in  the  ability  for  respondents  to  save  their  answers  at  any 
time,  close  the  survey,  and  return  later  to  complete  it. 

As  with  the  interview  effort,  the  Team  consolidated  the  individual  answers  into  a 
consensus  answer  for  each  of  the  twenty-four  questions.  Using  the  S02  methodology,  these 
responses  were  analyzed  to  determine  Sub-Issue  and  Issue  Findings.  Additionally,  the  team  used 
the  demographic  data  to  identify  logical  respondent  groups  based  upon  statistically  significant 
linkages  between  demographic  information  and  consensus  answers  identified  during  analysis. 
This  resulted  in  one  key  demographic  factor — level  of  military  experience — that  was  statistically 
significant  for  some  questions. 


Results 

The  first  step  taken  to  analyze  the  results  was  to  ensure  that  the  recorded  observations 
were  legible  and  intelligible,  and  that  they  were  organized  according  to  military  unit.  The 
experimenters  attempted  to  observe  a  cross-section  of  different  sized  units,  so  that  AARs 
reflected  a  diversity  of  unit  sizes  and  experience  levels.  Units  observed  included  on-screen  or  in- 
person  groups  ranging  from  approximately  25  to  100  Soldiers,  representing  ranks  that  included 
upper  level  noncommissioned  officers  and  commissioned  officers.  Within  the  various  observed 
groups,  one  officer  was  designated  as  a  battle  commander  (BC).  This  leader  generally  served  as 
an  intermediary  between  the  AAR  facilitator  and  the  training  group.  In  most  cases,  the  target 
AAR  was  conducted  mid-way  through  a  training  rotation. 

The  observational  data  were  then  transcribed  verbatim,  and  were  organized  according  to 
valence  (positive  or  negative  examples  of  witnessed  behavior)  and  specific  TTAAR  constructs 
represented.  Frequency  counts  were  then  tallied  to  determine  how  often  similar  observations 
were  made  of  facilitators  and  group  members  across  AARs.  The  findings  are  summarized  here. 
For  a  more  in-depth  view  of  the  data,  see  Mastaglio,  Wilkinson,  Bliss  and  Jones  (2010). 
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Observations.  Because  observed  groups  were  military  units,  we  observed  a  fairly  strict 
adherence  to  the  chain  of  command  by  the  trainees.  When  trainees  asked  questions  during  the 
AAR,  such  questions  often  were  directed  to  or  through  the  facilitator.  AAR  sessions  were 
generally  three  hours  long,  with  a  break  given  at  the  half-way  point.  Without  exception,  groups 
were  aware  that  the  AAR  sessions  were  being  recorded  for  concurrent  or  subsequent  review. 

This,  along  with  the  presence  of  commanding  officers,  may  have  served  to  restrict  the  volume 
and  content  of  the  questions  and  the  flow  of  information. 

It  is  important  to  stress  that  observed  AARs  reflected  formal  events,  not  informal  “hot 
washes”  at  the  training  site.  We  have  organized  our  findings  by  associating  observations  with 
construct  elements  within  the  ITAAR  model.  We  also  separated  our  observations  according  to 
their  relevance  to  the  facilitator  or  the  group  as  a  whole  (we  discuss  each  below).  In  addition,  we 
counted  both  the  positive  and  negative  behavior  observations  for  each  site. 

The  content  of  the  AAR  dialog  generally  occurred  at  a  high  level.  Rather  than  specific 
comments,  personnel  verbalized  goals  which  often  precluded  targeted  actions  for  remediation. 
Similarly,  much  of  the  commentary  represented  clarification  of  comments  or  problem  elements, 
not  the  generation  of  solution  and  alternatives.  In  cases  where  effective  dialog  occurred,  it 
tended  to  revolve  around  emotional  expression  (e.g.,  trust,  frustration,  or  surprise).  Generally, 
there  seemed  to  be  an  abundance  of  “ground  truth”  from  training  exercise  recordings.  What  was 
missing  was  summative  or  accurate  evaluative  commentary. 

Facilitator  Insights.  At  the  beginning  of  our  experimental  observation  periods,  we  were 
usually  told  that  facilitators  had  been  trained  to  conduct  AARs.  This  did  appear  to  be  the  case 
across  installations  and  situations,  as  facilitators  tended  to  follow  a  common  procedure.  That 
procedure  included  introductory  statements  clarifying  the  purpose  and  pedagogical  philosophy  of 
AARs,  listing  of  goals  to  be  achieved  during  the  AAR,  presentation  of  open-ended  questions 
designed  to  instruct  (Socratic  style),  presentation  of  “ground  truth”  information  designed  to 
enhance  Soldier  SA  and  particular  roles,  use  of  technology  media  to  encourage  cohesion, 
presentation  of  individual  awards,  and  encapsulation  of  goals  for  the  subsequent  training  period. 

Specific  observations  revealed  that,  as  a  whole,  instructors  appeared  to  be  effective  in 
accomplishing  their  goals.  Nine  positive  comments  indicated  that  they  were  generally  organized, 
centered  on  performance  improvement,  willing  to  relinquish  control  of  the  session  when 
appropriate  (often  to  the  BCs),  willing  to  positively  reinforce  effective  behaviors  and 
constructively  identify  ineffective  ones,  apt  to  use  personal  anecdotes  to  illustrate  pedagogical 
points,  and  fluent  with  technology  use. 

The  four  negative  facilitator  comments  noted  tendencies  to  ask  broad  or  leading 
questions,  focusing  on  task  issues  at  the  expense  of  individual  strengths  or  weaknesses,  and 
directing  comments  to  individuals  instead  of  collective  units. 

Group  Insights.  In  contrast  to  the  facilitator  comments,  group  behaviors  were  commonly 
judged  more  negatively.  In  fact,  of  the  eleven  observations  that  were  judged  to  be  universal 
across  AAR  groups,  eight  were  negative  and  three  were  positive.  The  positive  comments 
included  group  members  seen  as  knowledgeable,  focused  on  performance  goals,  and  eager  to 
provide  input  to  the  AAR  process.  However,  these  observations  were  outweighed  by  frequent 
indications  that  group  members  were  passive  (mostly  listening  and  providing  little  input), 
deferent  to  rank,  focused  on  personal  blame,  reluctant  to  criticize,  and  unwilling  to  interact  with 
the  visual  aid  technologies  available.  In  addition,  there  were  physical  aspects  of  the  AAR 
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environment  that  hindered  effective  dialog.  Those  included  high-ranking  individuals 
consistently  seated  toward  the  front  of  the  room  and  heterogonous  rank  representation  within  the 
AAR  group.  The  net  result  of  these  aspects  was  that  groups  generally  lacked  adequate  social 
interaction,  even  when  problem  solutions  required  group  problem  solving.  In  turn,  this 
influenced  facilitators  to  adopt  a  more  directive,  instructive  presentation  style. 

NTC  and  BCTP  Interview  Findings 

In  the  following  section  we  identify  each  issue  (Issue  1  through  3)  and  discuss  overall 
findings  at  that  level.  To  provide  insight  into  how  we  reached  the  issue  level  findings  next  are 
shown  the  Sub-Issues  and  findings  relative  to  them  based  on  our  analysis  of  respondent  answers. 

Issue  1:  How  are  AARs  considered  during  the  Design,  Planning,  and  Preparation  Phases? 

Issue  Finding :  Based  upon  the  identified  training  event  focus  areas,  facilitators  plan  the 
actions  of  the  facilitation  team  to  best  support  the  training  event.  This  includes  a  data  collection 
and  management  plan,  which  details  how  appropriate  assets,  both  other  facilitators  and 
instrumentation,  are  positioned  in  time  and  space  to  collect  critical  information  while  minimizing 
possible  disruptions  to  the  training  scenario.  Facilitators  prepare  by  meeting  their  counterpart(s), 
who  are  responsible  for  unit  performance  and  with  whom  they  will  most  often  interact.  Also 
important  during  the  pre-training  event  phase  is  the  establishment  of  a  positive  and  supportive 
relationship  between  the  facilitator  and  the  training  audience. 

1.1:  How  do  AARs  impact  the  design  and  planning  of  a  training  event? 

Consensus  Sub-Issue  Finding :  The  AAR  is  an  opportunity  to  temporarily  emerge  from 
a  training  scenario  and  its  stressors  to  conduct  a  comprehensive  review  of  a  small  number  of 
performance  requirements,  focusing  on  what  occurred  and  why. 

The  interviewed  respondents  represented  OC/Ts  from  two  of  the  Army’s  CTCs.  CTC 
training  is  highly  structured.  AARs  occur  following  the  conclusion  of  a  portion  of  the  training 
event.  Facilitators  have  no  input  into  when  this  AAR  occurs.  Therefore,  these  respondents  have 
little  if  any  opportunity  to  impact  the  sequencing  of  their  supported  training.  Respondents  are 
able  to  influence  training  content.  Their  content  changes  arise  from  training  unit  objectives  and 
current  operational  experiences  and  conditions. 

1.2:  How  do  AAR  practitioners  prepare  for  training  event  AARs? 

Consensus  Sub-Issue  Finding :  During  planning  and  preparation  for  a  training  rotation, 
facilitators  familiarize  themselves  with  the  scheduled  training  and  unit  training  objectives.  They 
use  unit  provided  materials,  such  as  standard  operation  procedures  and  past  unit  training 
assessments,  to  identify  a  general  body  of  performance  requirements  and  then  review  doctrine, 
tactics,  techniques,  and  procedures  (TTP),  and  operational  examples  to  hone  their  knowledge. 
AARs  are  conducted  within  a  formal  facilitation  structure  and,  thus,  guidance  from  the  O/C 
chain  is  critical  for  AAR  planning  and  preparation. 

Based  upon  identified  training  event  focus  areas  that  result  from  this  process,  facilitators 
plan  the  actions  of  the  facilitation  team  to  best  support  the  training.  This  includes  a  data 
collection  and  management  or  coverage  plan,  or  how  they  insure  that  appropriate  assets  are 
positioned  in  time  and  space  to  collect  critical  information  while  minimizing  disruptions  to  the 
training  scenario.  Assets,  including  subordinate  O/Cs,  instrumentation,  and  other  members  of 
the  facilitation  team,  are  positioned  to  record  “critical”  events,  these  are  defined  as  those 
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“friction  points”  that  most  impact  mission  accomplishment.  The  initial  plan  is  reviewed  daily  to 
measure  overall  training  success  and  to  redirect  exercise  events  and  observer  coverage  when 
required  to  insure  training  objectives  and  focus  areas  are  properly  supported.  Senior  O/C 
guidance  is  also  important  in  identifying  critical  events.  Materials  used  to  record  and  organize 
data  are  individualized  and  may  include  checklists,  graphics,  and  charts  as  well  as 
instrumentation,  especially  visual  collection  means,  the  latter  are  considered  particularly 
powerful. 

The  final  preparation  for  the  facilitator  is  meeting  their  counterpart(s),  defined  as  the 
individual(s)  who  are  responsible  for  unit  performance  and  with  whom  they  will  most  interact 
during  the  training  event.  This  relationship  is  important  and  should  consist  of  a  partnership  with 
the  facilitator  serving  as  an  extension  of  unit  leadership.  Also  important  during  the  pre-training 
event  phase  is  establishing  a  positive  and  supportive  relationship  between  the  facilitator  and  the 
training  audience. 

Issue  2:  How  are  AARs  considered  during  Execution  Phase? 

Issue  Finding:  O/Cs  prepare  for  the  AAR  through  a  series  of  preparation  activities  that 
take  place  during  training  event  execution,  such  as  identifying  focus  areas,  or  unit  performance 
considerations  for  attention.  For  example,  during  the  training  event,  data  collection  is  a  key  task 
to  support  AAR  preparation.  In  addition,  facilitators  identify  individuals  who  might  be  more 
receptive  to  training  and,  therefore,  better  conduits  for  facilitating  unit  improvement  during  and 
after  the  AAR.  They  will  also  monitor  unit  limitations  due  to  equipment  or  personnel  that  might 
result  in  an  adjustment  of  training  content  or  focus  areas. 

AAR  preparation  involves  reducing  the  data  collected  to  a  manageable  quantity  that  best 
supports  the  original  training  objectives.  Senior  mentors  and,  especially,  counterparts  are  the 
decision  authority  in  this  process.  In  addition,  facilitators  support  data  collection  and  the 
“filtering”  step  by  interacting  with  their  counterparts  DAR,  which  are  simple,  frequent,  and 
immediate  “hot  washes”  that  allow  collaboration  in  a  private  and  infonnal  manner.  Leveraging 
the  O/C  structure  allows  the  facilitator  to  achieve  a  breadth  and  depth  of  perspective,  identifying 
what  actually  occurred,  or  “ground  truth”,  and  why.  This  process  allows  the  facilitator  to  develop 
a  small  number  of  focus  areas  for  the  AAR,  usually  three  to  five,  though  possibly  more. 

The  goal  of  the  AAR  is  training  transfer,  represented  through  the  assignment  of 
responsibility,  changes  to  unit  performance  during  a  training  event,  and  capture  of  changes  in 
unit  documents,  such  as  SOP.  AAR  considerations  include  the  establishment  of  ownership  by 
the  training  audience  over  the  issues  and  improvements  identified  during  the  AAR.  Facilitators 
encourage  group  discussion,  ask  open-ended  questions,  and  practice  patient  silence  while 
refraining  from  hierarchical  criticism  and  teaching.  In  addition,  the  facilitator  also  seeks  to 
establish  credibility  with  the  training  audience. 

During  the  AAR,  facilitators  work  to  create  a  positive  and  comfortable  learning 
atmosphere,  promoting  both  physical  and  affective  comfort.  The  purpose  is  to  foster 
participation,  discussion,  openness,  self-discovery,  and  ownership  of  the  AAR  and  its  outcomes. 
The  facilitator  may  practice  the  behaviors  he  desires  from  the  training  audience,  such  as 
listening,  attention,  and  focus. 

Because  the  AAR  is  a  discussion,  a  facilitator  does  not  prevent  members  of  the  training 
audience  from  varying  excessively  from  doctrine  or  other  published  processes,  or  from 
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established  AAR  objectives  as  long  as  the  discussion  is  relevant  to  performance  improvement. 
During  these  discussions,  facilitators  serve  as  knowledge  providers,  citing  examples  from  either 
doctrine,  the  training  event,  or  sources  such  as  in-theater  experience  (i.e.,  a-ways). 

Facilitators  assess  the  effectiveness  of  the  AAR  through  group  participation  and 
discussion.  Non-verbal  cues  from  the  training  audience  will  help  the  facilitator  gauge  acceptance 
of  opportunities  to  learn  by  the  training  audience;  these  cues  include  eye  contact,  posture, 
attention,  and  note  taking.  Ultimately,  AAR  success  is  measured  by  the  training  audience 
acceptance  of  a  manageable  number  of  issues  and  assignment  of  responsibility  for  fixing  those 
issues.  This  can  be  assessed  based  upon  unit  efforts  to  propose  and  execute  agreed  upon  changes 
in  future  performance. 

2.1:  What  are  the  AAR  considerations  during  training  execution? 

Consensus  Sub-Issue  Findings:  During  the  training  event,  data  collection  is  a  key  task 
to  support  AAR  preparation.  In  addition,  facilitators  are  concerned  with  the  receptiveness  of 
training  unit  members  and  identify  individuals  who  might  be  more  receptive  and,  therefore, 
better  conduits  for  facilitating  unit  improvement.  Identifying  focus  areas,  or  unit  performance 
considerations  for  attention,  is  central  to  facilitators.  They  will  also  monitor  unit  limitations  due 
to  equipment  or  personnel,  which  might  result  in  an  adjustment  of  training  content  or  focus 
areas. 

Facilitators  interact  with  their  counterparts  in  during  action  reviews  (DAR),  or  “hot 
washes”  (also  called  “green  book  AARs”  or  “hood  top  AARs”).  They  generally  see  these  hot 
washes  as  critical,  sometimes  even  more  important  than  the  AAR.  The  hot  washes  are  important 
because  they  are  simple,  frequent,  and  immediate  and  because  they  allow  the  facilitator  to 
interact  with  their  counterpart  in  a  private  and  informal  manner.  DARs  may  also  be  conducted  to 
provide  immediate  feedback  to  trainees  as  long  as  doing  so  is  not  disruptive  to  the  training. 

2.2:  AAR  considerations  during  AAR  preparation 

Consensus  Sub-Issue  Finding :  The  AAR  preparation  process  starts  with  an  abundance 
of  information  and  insights  from  operational  and  training  instrumentation,  including  C4ISR 
system  data  and  video  recordings.  During  training,  facilitators  will  view  C4ISR  systems  at 
appropriate  times  to  capture  key  events;  they  must  also  maintain  flexibility  to  capture  unexpected 
yet  critical  key  events.  The  AAR  preparation  process  involves  reduction  of  the  data  to  a 
manageable  subset  that  best  supports  the  original  training  objectives.  Senior  mentors  and  their 
unit-leader  counterparts  are  the  decision  authority  in  this  process. 

Preparing  for  an  AAR  is  defined  as  actions  taken  between  the  end  of  observed  training 
and  the  beginning  of  a  specific  AAR.  It  is  often  accomplished  in  a  relatively  short  period  of 
time.  Facilitators  focus  preparation  on  identifying  what  actually  occurred,  or  “ground  truth”,  and 
why.  They  leverage  the  O/C  structure  to  achieve  a  breadth  and  depth  of  perspective,  using  their 
own  notes  plus  discussion  with  and  notes  from  other  O/Cs  and  from  instrumentation. 
Instrumentation  is  important  in  showing  ground  truth,  though  facilitators  warn  it  must  be  used  in 
a  positive  manner  as  training  audiences  tend  to  fear  that  instrumentation  generated  data  will  be 
used  as  evidence  against  them.  Facilitators  also  use  material  from  previous  AARs,  either  as 
building  blocks  for  the  AAR,  i.e.,  standard  questions  or  slides,  or  as  varied  models  of 
performance,  referred  to  as  “a-ways”. 
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This  AAR  preparation  is  used  to  develop  a  small  number  of  focus  areas  for  the  AAR, 
usually  three  to  five,  though  possibly  more.  Having  focus  areas  helps  the  facilitator  organize  and 
prepare  for  the  AAR,  even  though  facilitators  do  not  expect  the  AAR  to  cover  each  focus  area. 
DARs,  or  hot  washes,  are  integrated  into  AAR  preparation.  Facilitators  use  discussion  with 
counterparts  and  trends  identified  during  hot  washes  to  select  and  further  develop  key  focus 
areas.  They  rehearse  their  AAR  to  insure  they  are  familiar  with  the  material  to  be  presented  and 
to  coordinate  with  support  personnel  more  so  than  for  practicing  execution.  In  addition, 
facilitators  ensure  the  AAR  emphasizes  group  vice  individual  performance  by  focusing  on 
process  rather  than  individual  actions  as  well  as  using  plural  pronouns. 

2.3:  What  are  the  AAR  considerations  in  presenting  the  AAR? 

Consensus  Sub-Issue  Finding :  The  primary  consideration  during  presentation  of  an 
AAR  is  to  establish  training  audience  ownership  of  the  issues  and  improvements  identified 
during  the  AAR.  Ownership  is  defined  as  a  specific  strategy  for  improvement,  a  plan  to  achieve 
that  improvement,  and  responsibility  being  assigned  to  an  individual  for  execution  of  that  plan. 
Establishing  ownership  of  an  issue  and  improvement  plan  is  more  important  than  following  a 
schedule,  reviewing  a  set  number  of  focus  areas,  or  the  quality  or  depth  of  discussion  of  a  focus 
area.  The  goal  is  training  transfer,  represented  through  the  assignment  of  responsibility,  changes 
to  unit  performance  during  a  training  event,  and  capture  of  changes  in  unit  processes,  such  as 
SOPs. 

Facilitators  work  to  have  the  training  audiences  to  focus  on  “what  happened,  why  it 
happened,  how  to  fix  it,  and  who  is  going  to  fix  it,”  in  terms  of  collective  performance.  Group 
discussion  is  necessary  to  accomplish  this  and  facilitators  achieve  group  discussion  through  use 
of  open-ended  questions,  which  are  targeted  towards  the  group.  Open-ended  questions  targeted 
towards  specific  individuals  are  used  only  when  the  group  question  is  not  achieving  the  desired 
discussion.  Facilitators  also  use  patient  silence  to  evoke  training  audience  member  responses. 
They  develop  a  question  strategy  during  AAR  preparation  that  includes  follow-up  questions  and 
identification  of  which  individuals  should  answer  those  questions. 

Facilitators  work  hard  to  avoid  adversarial  hierarchical  situations.  For  this  reason,  they 
avoid  judgmental  approaches  and  directive  facilitation  or  “teaching.”  They  focus  their  efforts 
towards  the  group,  avoiding  individual  attribution.  They  do  this  by  focusing  on  unit  process  and 
on  what  happened  and  why,  avoiding  who.  They  also  use  plural  forms  and  group-based  verbal 
constructs.  Though  individual  attribution  may  be  desirable  (e.g.,  highlighting  positive  individual 
performance),  individual  attribution  of  a  negative  nature  is  sometimes  unavoidable.  When  this  is 
the  case,  facilitators  depend  upon  the  professionalism  of  the  training  audience  to  avoid  adverse 
impacts. 

O/C  credibility  with  the  training  audience  is  essential  to  effective  facilitation.  They 
establish  credibility  through  personal  experience  and  demonstrated  knowledge,  both  doctrinal 
and  situational.  Facilitators  are  subtle  in  advertising  their  experience,  being  aggressive  can 
trigger  a  defensive  attitude  in  their  counterparts  or  entire  training  units.  Uniform  patches  and 
badges  are  a  passive  means  to  advertising  experience.  Working  experience  into  informal 
discussion  is  a  more  active  means.  AAR  participation  is  severely  restricted  to  the  training 
audience  and  personnel  external  to  the  training  audience  are  usually  not  allowed  to  engage  in  the 
AAR. 
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2.4:  What  are  the  AAR  considerations  in  interacting  with  the  training  audience? 

Consensus  Sub-Issue  Finding :  In  the  AAR,  facilitators  work  to  create  a  positive  and 
comfortable  learning  atmosphere.  The  purpose  is  to  foster  participation,  discussion,  openness, 
self-discovery,  and  develop  ownership  of  the  AAR  and  its  results.  They  achieve  physical 
comfort  within  the  AAR  by  allowing  participants  to  remove  awkward  uniform  items.  They 
achieve  affective  comfort  through  an  emphasis  on  the  fact  that  the  AAR  belongs  to  the  unit  and 
is  a  learning  event.  They  may  also  use  humor  to  encourage  affective  comfort.  In  addition, 
facilitators  focus  on  active  listening,  demonstrating  expected  AAR  participant  behaviors,  and 
further  fostering  group  participation.  Facilitators  are  also  aggressive  at  note  taking,  as  visual 
notes  from  the  discussion  can  further  focus  the  discussion  and  again  demonstrate  to  the  training 
audience  attention  and  other  desired  behaviors. 

Facilitators  allow  training  audience  discussion,  as  long  as  it  is  relevant  to  improved  unit 
performance,  to  vary  freely  from  planned  topics  because  the  discussion  itself  is  an  objective  of 
an  AAR.  Their  experience  has  shown  that  free  discussion  normally  does  not  vary  significantly 
from  training  results  and  that  it  supplies  opportunities  for  the  facilitator  to  extract  learning  points 
that  support  AAR  objectives.  As  a  result,  when  the  training  audience  is  speaking,  facilitators 
remain  mostly  quiet  and  allow  the  discussion  to  progress.  Similarly,  the  facilitator  does  not 
prevent  members  of  the  training  audience  from  varying  excessively  from  doctrine  or  other 
published  processes.  Rather,  the  facilitator  focuses  the  AAR  on  self-discovery,  helping  the 
training  audience  talk  through  new  perspectives,  focus  on  cause  and  effect,  and  identify  how  the 
non-doctrinal  process  may  or  may  not  be  better  than  published  methods.  During  these 
discussions,  facilitators  serve  as  knowledge  providers,  citing  examples  either  from  doctrine,  the 
training  event,  or  other  sources  such  as  theater  experience  (i.e.,  performance  models  or  “a- 
ways”). 

It  is  recognized  that  training  groups  are  often  tiered.  The  first  tier  includes  primary 
participants  and  targeted  learners  of  the  AAR,  or  active  learners.  The  second  tier  primarily 
observes  and  learns  passively.  Often,  facilitators  will  direct  questions  to  the  second  tier  to 
generate  information  or  positions  within  the  AAR  discussion  to  foster  discussion  by  the  first  tier. 

Facilitators  assess  the  progress  of  the  AAR  using  the  amount  of  participation  and  group 
discussion  as  a  gauge.  They  also  look  for  non-verbal  cues  to  the  training  audience’s  acceptance 
of  the  learning  opportunity  through  eye  contact,  posture,  attention,  and  audience  note  taking. 
Should  the  facilitator  assess  that  the  AAR  is  not  progressing,  they  will  respond  by  moving  to  the 
next  focus  area.  Facilitators  avoid  confrontation  between  the  facilitator  and  the  training  audience 
by  pre-AAR  coordination  with  counterparts.  When  confrontation  does  occur,  they  use 
knowledge  of  doctrine,  TTPs,  and  SOPs  to  depersonalize  and  diffuse  the  situation.  Facilitators 
actually  avoid  participation  by  leadership  when  AARs  are  not  progressing  in  order  to  keep  their 
counterparts  neutral.  Ultimate  AAR  success  is  measured  by  the  acceptance  of  a  limited  number 
of  issues  and  responsibility  for  fixing  those  issues,  exhibited  by  unit  efforts  to  propose  and 
execute  performance  changes. 

Issue  3:  How  are  AARs  considered  during  the  Evaluation  Phase? 

Consensus  Issue  Finding :  Post- AAR,  facilitators  follow  through  by  using  the  AAR 
results  to  identify  ways  to  positively  impact  training  in  the  future.  The  following  three  strategies 
for  doing  this  were  identified  in  this  research: 
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•  reviewing  results  with  facilitator  counterparts 

•  socializing  results  with  fellow  O/Cs,  especially  superiors 

•  adjusting  future  training  events  to  provide  an  opportunity  for  the  training  audience  to 

practice  planned  improvements. 

Post- AAR  activity  is  also  centered  on  determining  metrics  of  training  transfer  for  future 
performance.  Facilitators  focus  on  identifying  these  unit  performance  changes  to  determine  if 
they  were  accepted  by  the  training  unit. 

AAR  results  also  produce  various  changes  to  AAR  facilitation  strategies  over  time.  As  a 
result,  facilitators  may  select  more  complex  and  challenging  areas  in  which  to  focus  the  AAR  in 
the  future.  One  example  is  an  effort  to  be  more  “patient”  and  to  emphasize  non-AAR 
interactions  more  over  time.  There  are  a  variety  of  methods  for  incorporating  AAR  results  into 
future  practices.  These  include  identifying  trends,  discussing  these  trends  with  other  O/Cs, 
comparing  trends  to  what  is  occurring  in  theater,  and  modifying  training  events  to  front  load 
associated  training  events  within  an  exercise  as  a  result.  In  addition,  “sterilized”  AAR  “a-ways” 
can  be  saved  for  use  with  future  training  audiences. 

3.1:  How  are  AAR  results  incorporated  in  immediate/short-term  changes  in  performance? 

Consensus  Sub-Issue  Finding :  Facilitators  and  training  audiences  emerge  from  the 
AAR  with  a  contract  for  improvement.  The  following  are  three  strategies  for  following  up  on 
AAR  results:  review  of  results  with  facilitator  counterparts;  socialization  of  results  with  fellow 
O/Cs,  especially  superiors;  and  possible  adjustment  of  future  training  events  to  provide  an 
opportunity  for  the  training  audience  to  practice  planned  improvements.  Post- AAR  activity  is 
also  centered  on  determining  the  level  of  training  transfer  in  terms  of  changes  in  performance. 
They  follow  through  on  identifying  unit  performance  changes  to  determine  if  these  are  really 
implemented. 

Facilitators  do  not  normally  follow  up  with  individual  members  of  the  training  audience 
regarding  participation  in  the  AAR  unless  the  individual  was  so  disruptive  or  adversarial  that  it 
significantly  impacted  the  AAR. 

3.2:  How  are  AAR  results  incorporated  in  long-term  changes  to  performance? 

Consensus  Sub-Issue  Finding :  AAR  results  produce  various  changes  to  AAR  strategies 
over  time.  They  may  also  cause  modification  to  training  events  or  the  need  to  select  more 
complex  and  challenging  AAR  foci.  One  example  is  an  effort  to  be  more  “patient”  and 
emphasize  non-AAR  interactions  more  over  time.  Another  is  the  modification  of  the  training 
event  itself. 

3.3:  How  are  AAR  results  incorporated  in  future  AAR  practices? 

Consensus  Sub-Issue  Finding :  There  are  a  variety  of  methods  for  incorporating  AAR 
results  into  future  practices.  These  include  identifying  trends,  discussing  these  trends  with  other 
O/Cs,  comparing  trends  to  what  is  occurring  in  theater,  and  modifying  training  events  to  front 
load  associated  training  events  within  an  exercise  as  a  result.  In  addition,  they  save  “sterilized” 
AAR  products  for  use  with  future  training  audiences. 

Issue  4:  Additional  insights  and  observations. 
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During  data  collection,  especially  the  interview  mode,  when  following  the  S02 
methodology  there  are  always  opportunities  for  additional  insights  relative  to  the  research  area 
and  overall  goals  of  the  project,  some  of  these  deriving  from  additional  information  provided  by 
subjects  interviewed.  Therefore,  we  try  to  capture  those  insights  and  observations  even  though 
they  do  not  always  fit  within  the  Issue/Sub-Issue  structure.  In  this  section  we  will  discuss 
relevant  results  from  our  interviews  which  add  depth  of  understanding  about  AAR  as  currently 
practiced. 

Facilitators  identified  a  number  of  challenges  present  in  AAR  facilitation  as  well  as 
important  skills  and  techniques  for  both  junior  and  senior  leaders  to  address  these  challenges. 
These  provide  suggested  areas  of  focus  for  AAR  facilitator  training. 

Obtaining  comprehensive  and  accurate  training  event  facts  and  data  is  a  significant 
challenge  for  the  AAR  facilitator,  as  well  as  filtering  these  observations  for  potential  issues  to 
incorporate  into  the  AAR.  Crafting  and  asking  open-ended  questions,  especially  a  question  that 
generates  group  discussion,  is  a  challenge  as  well  since  the  issue  must  be  “deep”  enough  to  be 
accepted  by  the  training  audience.  A  final  challenge  is  the  execution  of  identified  “fixes”  to 
future  performance. 

Both  junior  and  senior  leaders  should  focus  on  fostering  training  audience  participation, 
have  a  call  plan,  avoid  talking  too  much,  and  be  comfortable  waiting  for  the  training  audience  to 
fill  silences.  In  addition,  in  recognition  of  the  role  of  facilitator,  as  opposed  being  a  formal 
instructor,  junior  and  senior  leaders  should  learn  to  be  non-adversarial,  to  use  open-ended 
questions,  and  to  listen  and  allow  the  unit  to  talk.  Also  facilitators  should  learn  to  display 
appropriate  affective  behaviors,  including  professionalism,  confidence,  and  passion.  Senior 
leaders,  in  particular,  should  be  more  cognizant  of  the  big  picture,  generational  differences,  and 
prepared  to  incorporate  teaching  opportunities  into  AARs  as  needed. 

4.1  Which  steps  in  the  AAR  process  present  the  most  significant  challenges  and  describe  them? 

Consensus  Finding :  Obtaining  comprehensive  and  accurate  training  event  facts  and  data 
is  a  significant  challenge,  as  well  as  filtering  these  observations  for  potential  issues  that  can  be 
incorporated  into  the  AAR.  Crafting  and  asking  open-ended  questions,  especially  a  question  that 
generates  group  discussion,  is  a  challenge  as  well  since  the  issue  and/or  supporting  visualizations 
must  be  “deep”  enough  to  be  accepted  by  the  training  audience.  A  final  challenge  is  the 
execution  of  identified  “fixes”  to  future  performance. 

4.2  What  AAR  skills  or  techniques  would  you  teach  to  junior  leaders? 

Consensus  Finding :  Important  AAR  skills  or  techniques  for  junior  leaders  include  the 
following. 

•  fostering  training  audience  participation 

•  developing  and  using  a  call  plan 

•  avoiding  a  tendency  to  talk  too  much 

•  becoming  comfortable  with  waiting  for  the  training  audience  to  respond  to  periods  of 

silence 

Other  skills  include  not  becoming  adversarial,  the  use  of  open  ended  questions,  focusing 
on  basic  questions,  and  listening,  or  allowing  the  unit  to  talk.  Finally,  junior  facilitators  should 
learn  to  display  appropriate  affective  behaviors,  including  professionalism,  confidence,  and 
passion. 
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4.3:  What  AAR  skills  or  techniques  would  you  teach  to  senior  leaders? 

Consensus  Finding :  Senior  leaders  should  be  taught  the  same  skills  as  junior  leaders,  to 
foster  training  audience  participation,  e.g.,  by  minimizing  the  number  of  slides  used  for  an  AAR 
and  instead  focusing  on  unit  discussion.  Senior  leaders  should  be  more  cognizant  of  the  big 
picture,  generational  differences,  and  incorporate  more  teaching  into  AARs. 

Web-based  Survey  Findings 

Demographics.  The  survey  successfully  collected  responses  from  a  large  sample  of 
Soldiers,  dispersed  across  all  ranks  and  experience  levels.  In  total,  910  respondents  provided 
some  answers  to  the  survey  and  83 1  respondents  completed  the  entire  survey.  Respondents 
ranged  across  three  levels  of  experience:  231  were  Novice  Soldiers  with  0-7  years  of 
experience,  301  were  Mid-Career  Soldiers  with  8-15  years  of  experience,  and  378  were  Career 
Soldiers  with  15+  years  of  experience.  Respondents  also  came  from  all  Army  ranks.  The  spread 
of  ranks  is  close  to  actual  Army  demographics  as  of  FY  2008  (2)  (Maxfield,  2008). 

The  majority  of  respondents  described  themselves  as  having  at  least  some  experience 
leading  or  facilitating  AARs  (89.97%).  Respondents  reported  leading  an  average  of  13.65  AARs 
within  the  last  twelve  months;  compared  with  participating  in  an  average  of  20.84  AARs  over 
that  year. 

Table  3.  Spread  of  Soldier  Rank 


Rank 

Respondents 

Survey 

Demographics 

Army 

Demographics 

Specialist 

121 

71.16% 

83.76% 

NCO 

324 

Senior  NCO 

199 

Warrant  Officer 

10 

2.53% 

2.7% 

Chief  Warrant  officer 

13 

Company  Grade 

126 

26.29% 

13.51% 

Field  Grade 

107 

General  Officers 

5 

Survey  consensus  answers  and  findings.  In  the  following  section  we  describe  the 
results  of  our  analysis  of  respondent  answers  and  the  resulting  findings  for  Issues  1  and  2  and 
their  sub-issues.  Again,  since  the  consensus  answers  reflect  sub-issue  findings  which  in  turn 
aggregate  to  form  an  issue  finding  there  is  some  degree  of  redundancy  in  this  discussion  and  the 
reader  is  encouraged  to  review  the  subordinate  discussions  if  they  are  interested  in  how  overall 
findings  were  derived. 

Issue  1:  How  are  AARs  considered  during  the  Design,  Planning,  and  Preparation  Phases? 

Respondents  reported  that  the  AAR  had  a  role  within  training  but  that  role  was  not 
central.  Respondents  preferred  shorter,  more  frequent  AARs  or  other  performance  reviews. 
Respondents  reported  that  performance  models — how  other  units  executed  similar  missions — 
were  the  most  powerful  AAR  tool,  with  models  from  theater  operations  possessing  the  greatest 
legitimacy. 
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Sub-Issue  1.1:  How  do  AARs  impact  the  design  and  planning  of  a  training  event? 

Respondents  indicated  that  the  AAR  has  a  role  within  training,  though  not  a  central  role. 
They  reported  that  the  AAR  provides  less  than  50%  of  the  training  value  compared  with  other 
aspects  of  a  training  event.  Of  four  potential  methods  of  reviewing  performance,  respondents 
thought  shorter,  more  frequent  AARs  were  the  most  effective  but  also  thought  self  reflection  and 
face-to-face  discussion  with  OC/T  or  evaluator  were  effective.  Respondents  reported  that  longer, 
infrequent  AARs  were  less  effective. 

Higher  ranking  and  more  experienced  Soldiers  see  more  value  to  the  AAR.  Individuals 
who  consider  themselves  AAR  experts  also  view  the  AAR  as  a  more  important  aspect  of 
training. 

Question  1.1.1:  How  much  of  the  ultimate  training  value  of  a  training  event  comes  from  AARs 
(Less  than  25%,  25-50%,  51-75%,  More  than  75%)? 

The  AAR  provides  less  than  50%  of  the  value  in  a  training  event,  based  on  all  respondent 
answers.  More  than  60%  of  respondents  rated  the  AAR  as  providing  half  or  less  of  the  value  in 
the  training  event.  Only  23.2%  felt  the  AAR  provided  51-75%  of  the  training  value  and  only 
10.1%  believed  the  AAR  provided  more  than  75%. 

Question  1.1.2:  In  the  training  event,  how  important  is  the  AAR  compared  to  other  aspects  of 
the  training  event,  such  as  mission  rehearsal,  informal  feedback,  or  real  time  guidance  (1-Less 
important,  3-Equally  important,  5-More  important)? 

Overall,  respondents  indicated  that  the  AAR  is  less  valuable  than  other  aspects  of  the 
training  event  (M  =  2.09).  Over  80%  of  respondents  rated  the  AAR  as  3  or  less  (3  =  equally 
important)  when  compared  to  other  aspects  of  the  training  event.  However,  there  was  a  positive 
relationship  between  respondent’s  years  in  the  Army  and  self-described  proficiency  and  the 
value  of  the  AAR  compared  to  other  aspects  of  training.  This  relationship  indicates  that 
individuals  with  more  years  of  experience  or  a  higher  self-rated  proficiency  rated  the  AAR 
higher  than  respondents  with  fewer  years  of  experience  or  a  lower  self-rated  proficiency. 

Question  1.1.3:  How  important  is  each  of  the  following  for  reviewing  team  or  organizational 
performance  (1-No  impact  upon  training  event,  2-Minor  contribution  to  training  event,  3- 
Contributes  to  training  event,  4-Strongly  contributes  to  training  event,  5-Vital  to  training  event): 

•  Individual  self  reflection  by  the  members  of  the  team 

•  One  to  one  discussion  with  an  O/C  or  evaluator 

•  Frequent,  short,  informal  AARs 

•  Less  frequent  longer  and  more  formal  AARs 

Of  the  four  methods  presented  for  respondent  review,  frequent  short  informal  AARs 
received  the  highest  overall  rating  (average  of  3.49).  Rank  and  years  of  experience  also  had  an 
effect  on  this  question,  with  higher  ranking  Soldiers  and  those  with  more  years  of  experience 
rating  frequent,  short,  informal  AARs  higher  than  lower  ranking  Soldiers.  However,  one  to  one 
discussion  and  individual  self-reflection  also  received  average  ratings  over  3  (3.41  &  3.28 
respectively),  indicating  respondents  rated  all  of  these  methods  as  greater  than  “contributes  to  the 
training  event”  on  average.  Less  frequent  longer  and  more  formal  AARs  received  the  lowest 
overall  rating  (2.65  average),  indicating  respondents  tended  to  view  this  method  as  less  desirable. 
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Table  4.  Summary  of  Responses  to  “How  Important  Are  Each  Of  The  Following  For  Reviewing 
Team  or  Organizational  Performance” 


N 

Mean 

Individual  self  reflection  by  the  members  of  the  team 

775 

3.28 

One  to  one  discussion  with  an  O/C  or  evaluator 

775 

3.41 

Frequent  short  informal  AARs 

775 

3.49 

Less  frequent  longer  and  more  formal  AARs 

775 

2.65 

Valid  N  (listwise) 

775 

Question  1.1.4:  Of  the  previous  four  methods  of  reviewing  team  or  organizational  performance 
which  do  you  think  is  the  most  effective  (select  one)? 

•  Individual  self  reflection  by  the  members  of  the  team 

•  One  to  one  discussion  with  an  O/C  or  evaluator 

•  Frequent,  short,  informal  AARs 

•  Less  frequent  longer  and  more  formal  AARs 

Of  the  four  previous  methods  frequent,  short,  informal  AARs  are  the  most  effective. 
Respondents  selected  shorter  AARs  as  the  most  effective  method  over  50%  of  the  time,  with  no 
other  response  being  chosen  more  than  20%  of  the  time.  The  large  percentage  of  respondents 
who  selected  shorter  AARs  as  the  most  effective  of  the  four  methods  indicates  the  desire  for  less 
time  consuming  reviews  by  training  audience  members. 

Question  1.1.5:  What  should  be  the  atmosphere  within  an  AAR  (select  one)? 

•  Serious 

•  Relaxed 

•  Entertaining 

•  Formal 

•  Casual 

Respondents  overwhelming  opined  that  the  atmosphere  within  the  AAR  should  be 
relaxed  (62.2%).  No  other  answer  was  selected  more  than  22%  of  the  time. 

Sub-Issue  1.2:  How  do  AAR  practitioners  prepare  for  training  event  AARs? 

Respondents  rated  all  preparation  methods  as  important.  Performance  models — how 
other  units  have  executed  similar  missions — are  considered  the  most  powerful  tool  for  preparing 
AARs,  with  models  from  theater  possessing  the  greatest  legitimacy. 

Question  1.2.1:  How  important  is  material  from  the  following  sources  (1-Not  important,  3- 
Important,  5 -Very  important): 

•  Field  manuals  and  other  doctrine 

•  Examples  of  how  other  units  performed  the  same  mission 

•  Examples  of  how  the  same  mission  is  being  performed  in  theater 

•  Personal  examples  by  the  AAR  facilitator 
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Respondents  indicated  that  the  most  important  source  for  AAR  material  is  examples  of 
how  the  same  mission  has  been  performed  by  other  units,  especially  how  it  has  been  performed 
in  theater  (average  rating  of  4.31).  All  sources  were  considered  important,  with  no  other 
response  receiving  an  average  rating  above  3.6.  Higher  ranking  Soldiers  rated  examples  of  how 
other  units  are  performing  the  same  mission  higher  than  lower  ranking  Soldiers. 

Issue  2:  How  are  AARs  considered  during  Execution  Phase? 

The  respondent  AAR  participant  sample  perceived  the  AAR  as  a  continuation  of  the 
interaction  between  the  training  facilitator  and  the  training  audience  or  leaders.  They  saw  the 
role  of  the  facilitator  as  being  more  participative  than  directive  and  believed  that  the  AAR 
participants  should  have  a  key  role  in  the  direction  and  content  of  the  AAR,  which  should  serve 
as  providing  information  to  the  training  audience  in  terms  of  feedback.  Respondents  emphasized 
the  importance  of  the  experiences,  professionalism,  and  preparation  of  the  facilitator. 

Sub-Issue  2.2:  What  are  the  AAR  considerations  during  AAR  preparation? 

Respondents  indicate  that  a  large  majority  of  Soldiers  do  not  prepare  for  participation  in 
an  AAR,  and  of  those  who  do  there  are  no  predictable  patterns  that  can  be  leveraged  by  an  AAR 
facilitator.  This  emphasizes  the  importance  of  pre-AAR  preparation  by  the  AAR  facilitator  as  he 
or  she  is  likely  to  be  the  only  individual  organizing  thoughts  and  impressions  from  the  training 
execution. 

Respondents  believed  that,  although  not  essential,  it  was  important  to  incorporate 
material  from  all  during-event  reviews  into  the  AAR,  indicating  their  belief  that  the  AAR  is  a 
continuation  of  an  overall  performance  review  paradigm. 

Question  2.2.1:  How  do  most  Soldiers  prepare  for  an  AAR  (select  one)? 

•  Most  Soldiers  don’t  prepare  for  or  think  about  the  AAR 

•  Most  Soldiers  reflect  on  their  individual  performance  to  prepare  to  defend  themselves 

•  Most  Soldiers  reflect  on  what  other  members  of  their  unit  did  during  training 

•  Most  Soldiers  think  about  lessons  learned  from  the  training  session 

•  Most  Soldiers  reflect  upon  team  performance  to  be  prepared  for  the  discussion  in  the 

AAR 

Although  a  significant  number  (43%)  of  respondents  reported  that  their  fellow  Soldiers 
do  not  prepare  for  participation  in  an  AAR,  the  majority  of  respondents  do  believe  Soldiers 
conduct  some  preparation  for  the  AAR.  Of  the  respondents  who  indicated  that  Soldiers  do 
prepare  for  the  AAR,  they  were  evenly  divided  between  the  four  available  methods  for 
preparing. 

Question  2.2.2:  In  an  AAR  how  important  is  it  to  include  discussions  and  other  reviews 
conducted  prior  to  the  AAR  (1-Less  important,  3-Important,  5-More  important)? 

Respondents  indicated  that  it  is  important  to  include  discussions  and  other  during  action 
reviews  within  an  AAR,  but  not  essential.  Respondent  answers  fell  into  a  nearly  perfect  bell 
curve  for  this  question,  with  the  highest  and  lowest  possible  answers  having  nearly  the  same 
percent  selection  (9.3%  and  6.7%  respectively),  the  middle  answers  having  almost  identical 
percent  selection  (2  =  18.5%  and  4  =  21.2%)  and  the  middle  answer  (3)  receiving  the  highest 
percent  selection  (44.3%). 
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Sub-Issue  2.3:  What  are  the  considerations  in  presenting  the  AAR? 

TC  25-20  states  that  the  AAR  is  a  “professional  discussion”,  a  more  participative  or 
facilitative  act.  It  also  states  that  the  AAR  provides  “feedback  and  insight”  and  is  the  “keystone 
of  the  evaluation  process”,  feedback  and  evaluation  are  more  directive  acts.  Between  these 
somewhat  inconstant  purposes,  a  small  majority  of  respondents  identified  providing  feedback  as 
a  purpose  of  the  AAR,  implying  it  should  be  the  primary  driver  of  the  AAR  presentation. 

Respondents  reported  strong  beliefs  on  the  relationship  of  the  facilitator  to  the  training 
audience.  Respondents  indicated  that  it  is  important  to  very  important  to  incorporate  the 
opinions  of  the  training  audience  within  the  AAR.  Their  opinion  is  that  the  facilitator  is  a  partner 
to  the  training  audience  within  the  learning  process,  as  opposed  to  serving  in  a  more  directive 
role  where  they  represent  the  training  audience’s  chain  of  command.  In  regards  to  the  provision 
of  expertise,  respondents  hold  that  the  primary  source  of  expertise  within  the  AAR  is  the  training 
audience  and  that  the  central  role  of  the  facilitator  is  to  help  extract  the  tacit  expertise  of  the 
audience. 

To  establish  facilitator  credibility,  respondents  reported  that  personal  experience  was  the 
most  important  factor.  Professionalism  and  doctrinal  knowledge  were  also  important.  Rank  and 
branch  were  less  important.  Personal  experience  is  best  established  prior  to  the  execution  of  the 
AAR,  as  is  credibility  in  general.  Within  the  AAR,  it  is  important  to  display  professional 
behavior  and  personal  knowledge. 

According  to  respondents,  facilitators  should  plan  for  the  AAR  flow,  but  should  be 
prepared  to  allow  the  training  audience  to  vary  from  that  plan  when  that  variance  contributes  to 
overall  training. 

Question  2.3.1:  Below  are  several  possible  uses  for  video,  audio,  pictures,  graphics,  etc  within 
the  AAR.  Please  indicate  the  value  of  video,  audio,  pictures,  graphics  etc  for  accomplishing 
each  purpose  (1-Not  useful,  3-Somewhat  useful,  5-Very  useful): 

•  Providing  a  brief  overview  of  the  training  event  at  the  beginning  of  the  AAR 

•  Demonstrating  the  actual  performance  of  the  training  audience  during  portions  of  the 
training  event 

•  Helping  the  training  audience  get  back  on  track  when  discussing  what  occurred  during 
the  training  event 

•  Demonstrating  an  exemplar  way  to  complete  the  training  event  or  a  portion  of  it 

Across  the  board,  respondents  stated  that  media  were  useful  within  the  AAR,  though  the 
average  usefulness  score  ranged  between  3.49  and  3.75,  slightly  higher  than  “somewhat  useful”. 
Of  the  four  possible  uses,  helping  the  training  audience  get  back  on  track  was  the  highest  rated 
response  (3.75  average).  The  reported  usefulness  of  media  to  help  the  training  audience  get  back 
on  track  was  even  stronger  among  more  experienced  respondents.  More  experienced 
respondents  also  reported  asignificantly  higher  usefulness  of  media  to  provide  an  initial  overview 
than  did  less  experienced  respondents. 

Question  2.3.2:  Which  of  the  below  statements  is  more  accurate: 

•  Within  an  AAR,  the  facilitator  has  the  expertise  and  should  use  that  expertise  for 
assessing  and  improving  performance 
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•  Within  an  AAR,  the  training  audience  possess  the  required  expertise  and  should  be 
encouraged  and  supported  in  using  it 

Respondents  very  strongly  (76.3%  to  23.7%)  reported  that,  within  an  AAR,  the  training 
audience  possesses  the  expertise  required  to  understand  and  learn  from  the  demonstrated 
performance.  This  respondent  opinion  reinforces  the  perspective  that  the  primary  role  of  the 
AAR  facilitator  is  to,  indeed,  facilitate  the  discussion  among  the  training  audience. 

Question  2.3.3:  Which  statement  is  more  accurate: 

•  The  purpose  of  the  AAR  is  to  provide  feedback 

•  The  purpose  of  the  AAR  is  to  conduct  professional  discussion 

•  The  purpose  of  the  AAR  is  to  evaluate  performance 

Respondents  were  reasonably  evenly  divided  over  whether  the  purpose  of  the  AAR  is  to 
provide  feedback  (43.7%),  conduct  professional  discussion  (26.1%),  or  evaluate  performance 
(30.2%)  ;  the  leading  response  was  that  the  purpose  of  the  AAR  was  to  provide  feedback. 
Providing  feedback  implies  that  the  AAR  leader,  the  facilitator,  serves  in  as  an  expert  and  his/her 
role  is  more  instructional  or  evaluative  than  it  is  facilitative.  This  belief  is  counter  to  earlier 
reported  belief  that  the  training  audience  vice  the  facilitator  possesses  the  expertise  requisite  to 
the  AAR. 

Question  2.3.4:  Which  statement  is  more  accurate: 

•  The  AAR  facilitator  is  the  representative  of  the  training  unit’s  chain  of  command  and 
works  for  that  chain  to  help  the  training  unit 

•  The  AAR  facilitator  is  a  partner  with  the  training  unit’s  leader  and  works  with  the  unit 
leader  to  achieve  the  unit’s  training 

Respondents  very  strongly  (68.5%  to  31.5%)  view  the  role  of  the  facilitator  is  a  partner  to 
the  training  audience  leader  versus  a  representative  of  the  training  audience’s  chain  of  command. 
This  result  is  in  direct  opposition  to  role  implied  in  current  AAR  doctrine,  which  designates  the 
O/C  as  a  representative  of  the  command. 

Question  2.3.5:  How  important  are  the  following  factors  in  establishing  the  credibility  of  the 
facilitator  (1-Less  important,  3-important,  5-More  important): 

•  Personal  experience  of  the  facilitator 

•  Rank  of  the  facilitator 

•  Branch  of  the  facilitator 

•  Professionalism  of  the  facilitator 

•  Demonstrated  doctrinal  knowledge  of  the  facilitator 

Respondents  reported  that  the  professionalism  was  the  most  important  factor  (4.27  on  a 
scale  of  1  to  5)  in  establishing  the  credibility  of  the  AAR  facilitator,  followed  closely  by  the 
demonstrated  doctrinal  knowledge  (3.94  on  a  scale  of  1  to  5)  and  personal  experiences  (3.78). 
Branch  and  rank  were  not  considered  important  (2.55  and  2.30  on  a  scale  of  1  to  5,  respectively). 
Higher  ranking  and  more  experienced  respondents  had  a  significantly  higher  opinion  of  rated 
demonstrated  doctrinal  knowledge  higher  than  lower  ranking  or  less  experienced  respondents. 
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Question  2.3.6:  When  conducting  the  AAR  how  important  is  it  to  incorporate  the  training 
audience  member’s  perspectives  1-Less  important,  3-Important,  5-More  important)? 

Respondent  answers  fell  almost  exclusively  beyond  the  equally  important  rating  (3)  for 
this  question.  Respondents  strongly  believe  that  incorporating  the  perspectives  of  the  training 
audience  is  important  (97.8%)  with  a  significant  number  believing  it  is  very  important  (43.8%). 

Sub-Issue  2.4:  What  are  the  AAR  considerations  in  interacting  with  the  training  audience? 

The  related  questionnaire  asked  respondents  to  assess  a  variety  of  potential  facilitator 
AAR  behaviors.  The  survey  identified  nine  general  behaviors,  including  the  traditional 
facilitator  behavior  of  asking  open  ended  questions.  All  nine  were  rated  as  important.  However, 
identifying  what  actually  occurred  within  the  training  event,  using  examples  or  performance 
models  from  current  operational  theaters,  using  video  to  show  unit  actions,  and  ensuring  there  is 
open  discussion  were  rated  as  the  most  important,  in  sequence. 

Five  behaviors  were  associated  with  encouraging  group  participation  and  four  with 
controlling  group  discussion.  For  encouraging  participation,  asking  follow-on  questions  was 
rated  the  most  effective,  higher  than  asking  open-ended  questions  to  individuals  or  the  group  in 
general,  though  these  were  also  rated  effective  behaviors.  Respondents  rated  directing 
individuals  to  comment  on  other  participant’s  answers  and  remaining  silent  to  cause  group 
comment  as  less  effective. 

Respondents  indicated  a  perceived  importance  of  allowing  the  training  audience  a  level 
of  control  over  the  length  and  content  of  discussion  within  the  AAR.  Respondents  believe  the 
facilitator  should  occasionally  let  the  training  audience  discussions  vary  from  his  plan.  This  was 
also  reflected  in  the  respondents’  answer  to  what  behaviors  facilitators  should  exhibit  when  the 
training  audience  was  talking,  they  strongly  supported  some  involvement  but  were  evenly 
divided  between  some  correction  of  training  audience  input  and  the  a  more  directed  approach  of 
adding  to  the  training  audience  input. 

Question  2.4.1:  When  the  training  audience  is  providing  input  such  as  when  they  are  providing 
sustain  and  improve  comments  what  is  the  role  of  the  facilitator  (select  one)? 

•  The  facilitator  should  remain  quiet  and  let  the  training  audience  discuss  the  issue  among 
themselves 

•  The  facilitator  should  provide  some  guidance  if  the  training  audience  is  incorrect  in  their 
input 

•  The  facilitator  should  be  involved  in  the  input  and  adding  to  what  the  training  audience 
says 

•  The  facilitator  should  comment  on  every  piece  of  input  from  the  training  audience 

Respondents  believed  that  the  facilitator  should  engage  in  some  level  of  active 
involvement  in  training  audience  discussions  while  remaining  separate  from  that  discussion. 

Very  few  respondents  believed  in  the  upper  or  lower  level  description  of  various  facilitator 
participation  strategies,  remaining  quiet  (13.8%)  or  actively  engaged  in  the  discussion  through 
aggressive  engagement  (4.3%).  Respondents’  selection  was  close  to  evenly  split  between 
facilitators  providing  some  guidance  to  the  training  audience  (40.4%)  and  facilitators  being 
involved  in  the  input  and  adding  to  what  the  training  audience  says  (41.6%). 
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Question  2.4.2:  How  useful  are  each  of  the  following  actions  for  helping  organizations  benefit 
from  the  AAR  (1-Not  useful,  3-Somewhat  useful,  5-Very  useful)? 

•  The  facilitator  asks  open  ended  questions 

•  There  is  open  discussion  amongst  the  training  audience 

•  The  facilitator  encourages  AAR  participation  by  calling  on  specific  individuals 

•  The  facilitator  refers  to  published  doctrine  i.e.  FMs,  CALL  bulletins,  etc. 

•  The  facilitator  coaches  or  mentors  the  training  audience 

•  The  facilitator  identifies  what  actually  occurred  in  a  training  event,  i.e.  ground  truth 

•  The  facilitator  uses  video  or  audio  to  show  actual  unit  performance 

•  The  facilitator  shows  examples  of  how  other  units  executed  the  training  mission 

•  The  facilitator  uses  examples  from  in  theater  Iraq  or  Afghanistan 

All  of  the  provided  facilitator  behaviors  were  reported  by  respondents  as  being  more  than 
somewhat  useful.  The  most  useful  behaviors  were  using  identifying  what  actually  occurred 
within  the  training  event  (4.30),  using  examples  from  active  theaters  of  operations  (4.25),  using 
video  or  audio  to  show  actual  unit  performance  (3.97),  and  ensuring  there  is  open  discussion 
amongst  the  training  audience  (3.93). 

Question  2.4.3:  How  effective  are  the  following  techniques  for  encouraging  group  participation 
(1-Not  effective,  2-Partially  effective,  5-Very  Effective): 

•  Direct  open  ended  questions  to  individuals 

•  Direct  individuals  to  comment  on  other  individuals  answers 

•  Direct  open  ended  questions  to  the  group 

•  Ask  follow  on  questions 

•  Remaining  silent  after  asking  a  question  until  the  group  comments 

Of  the  potential  methods  for  encouraging  group  participation,  respondents  reported 
asking  follow-on  questions  as  the  most  effective  (4.13).  Further,  asking  follow-on  questions  was 
rated  very  effective  by  the  largest  number  of  respondents  (39.1%).  Asking  open  ended  questions 
to  individuals  and  to  the  group  were  reported  as  being  partially  effective  (3.64  and  3.63, 
respectively).  Directing  individuals  to  comment  on  another’s  answer  and  using  silence  to  cause 
group  comments  were  rated  as  less  than  effective  (2.91  and  2.62,  respectively). 

Question  2.4.4:  What  is  the  most  effective  strategy  for  directing  the  training  audience 
discussion  during  the  AAR  (select  one)? 

•  Let  the  training  audience  completely  drive  the  discussion 

•  Occasionally  let  the  training  audience  vary  if  the  discussion  is  productive 

•  Try  to  keep  the  training  audience  on  your  planned  discussion  path 

•  Never  let  the  training  audience  vary 

Half  (50.1%)  of  the  respondents  reported  that  occasionally  letting  the  training  audience 
vary  from  planned  discussion  was  the  most  effective  strategy  for  directing  discussion  during  an 
AAR.  Generally,  respondents  rejected  letting  the  training  audience  completely  drive  discussion 
(10.4%)  and  never  letting  the  training  audience  vary  (4.2%).  One  third  (34.7%)  of  respondents 
supported  a  more  aggressive  effort  by  facilitators  to  control  the  AAR  discussion.  The  bottom 
line  is  that  respondents  (95.8%)  do  not  believe  in  a  highly  controlled  AAR  experience. 
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Discussion 


AAR  doctrine  describes  a  discrete  event  that  occurs  after  some  amount  of  significant 
training  has  been  completed.  This  doctrine  describes  the  AAR  as  if  it  were  something  that  occurs 
external  to  the  training.  Our  examination  of  AAR,  on  the  contrary,  shows  that  common  practice 
and  Soldier  attitudes  reflect  a  full  assimilation  of  the  AAR  within  and  as  a  key  part  of  the 
training. 

Common  Practice 

The  AAR  has  become  one  element  of  the  overall  interaction  between  the  trainer  (and 
training  system)  and  the  training  audience.  Its  boundaries  within  that  interaction  have  been  open 
to  individual  interpretation.  This  has  raised  disagreement  between  AAR  purists  who  want  to  stay 
with  the  original  interpretation  of  the  AAR  and  practitioners  who  loosely  use  the  term  “AAR” 
both  as  noun  and  verb  to  describe  any  performance  review.  For  the  most  part,  this  report  uses 
“AAR”  in  its  doctrinal  sense  as  an  organized  and  facilitated  group  review/discussion  following  a 
significant  training  activity.  We  used  other  common  terms,  such  as  “hot  wash”,  to  describe  other 
performance  review  efforts  and  further  propose  that  the  Army  should  increase  use  of  the  term 
DAR  to  capture  these  interim  review  events  that  do  not  fully  meet  the  doctrinal  definition  of  an 
AAR. 

The  AAR  is  a  multi-disciplinary  activity  drawing  from  a  number  of  the  human  sciences. 
Traditionally,  AARs  have  been  associated  with  learning  sciences.  Learning  sciences  remain 
integral  to  the  AAR  but  the  attitudes  and  practices  of  AAR  practitioners  show  that  the  AAR  is 
also  quite  dependent  on  other  cognitively-oriented  fields  of  expertise  such  as  knowledge 
management  and  leadership. 

A  key  observation  from  this  research  is  the  importance  of  AAR  acceptance  by  members 
of  the  training  audience.  A  majority  of  the  O/C  effort  is  targeted  towards  achieving  this 
acceptance,  including  a  de-emphasis  of  the  number  focus  areas  and  an  emphasis  on  O/C 
credibility,  peer  relationships,  acceptable  ground  truth,  participation,  and  free  discussion.  O/Cs 
adjust  their  verbal  and  non-verbal  communication  patterns  to  foster  participation  and  acceptance. 
This  could  in  part  be  because  O/Cs  are  external  to  the  training  group.  They  have  no  formal 
authority  over  that  group  during  training  and  no  authority  after  training  is  complete. 

O/Cs,  who  believe  they  have  the  ability  to  help  a  unit  improve,  and  want  to  help  a  unit 
preparing  to  deploy  to  a  theater  of  operations,  must  use  persuasion  and  example  to  influence 
group  behaviors.  O/Cs  usually,  as  a  result  of  their  own  experience,  have  sufficient  expertise  that 
they  will  be  successful  in  using  persuasion  and  example  in  an  effective  manner.  Many  of  the 
processes  they  use  rely  upon  subtlety,  such  as  passively  or  indirectly  citing  their  own  experience 
and  knowledge  credentials,  or  their  ability  to  use  silence  to  foster  response. 

The  second  key  observation  is  the  emphasis  placed  on  DAR  or  hot  washes,  relative  to  the 
AAR.  One  O/C  opined  that  hot  washes  were  actually  more  critical  to  training  success  than 
AARs.  This  could  be  due  to  several  factors.  The  Situation  Training  Exercise  (STX)  lends  itself 
to  the  use  of  frequent,  simple,  and  relatively  easy-to-develop  hot  washes.  Such  reviews  also 
develop  stronger  partnerships  between  the  trainer  and  the  training  audience  than  a  more  formal 
AAR  structure.  The  potential  issue  with  these  reviews  is  the  inability  to  draw  on  data  that 
establish  “ground  truth”,  and  instead  assume  “ground  truth”  based  on  information  readily 
available  at  the  time, which  is  often  incomplete  or  incorrect.  Therefore,  learning  may  be  flawed 
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because  it  becomes  based  on  incomplete  assumptions  or  incorrect  information  about  what 
actually  occurred. 

The  third  point  of  interest  from  these  interviews  is  the  change  in  relationships  found 
within  the  AAR.  Figure  2  depicts  the  traditional  AAR  relationship,  as  described  in  the  limited 
AAR  doctrine.  It  is  a  relationship  between  the  facilitator  (fa)  and  the  training  audience  (ta)  and 
where  the  facilitator  is  implied  to  be  an  extension  of  the  training  audience’s  superior.  In  this 
relationship,  the  facilitator  or  O/C  leads  the  AAR  (FM7-1,  2003). 


Figure  2.  Traditional  AAR  participant  relationship. 


The  NTC  O/Cs  interviewed  describe  a  different  relationship.  In  this  relationship,  as 
depicted  in  Figure  3,  the  facilitator  is  partnered  with  their  counterpart  (cp)  in  supporting  the 
training  audience’s  learning. 


Figure  3.  Reported  NTC  AAR  participant  relationship. 

BCTP  O/Cs  must  deal  with  an  additional  participant,  the  senior  mentor  (figure  4).  The 
senior  mentor  is  not  accounted  for  in  AAR  doctrine.  He  is  a  normally  a  retired  senior  officer.  In 
situations  where  the  counterpart  significantly  outranks  the  facilitator,  a  senior  mentor  adds  the 
credibility  necessary  to  achieve  an  effective  AAR.  However,  the  senior  mentor  complicates 
facilitator-counterpart-audience  relationship.  From  our  observations  of  the  role  of  senior 
mentors  (sm)  during  this  investigation,  it  appears  that  he/she  fills  the  position  doctrines  describes 
for  the  traditional  O/C,  someone  positioned  over  the  training  audience  including  the  counterpart 
leadership. 


Figure  4.  Senior  mentors  relationship  to  training  audience. 

The  fourth  observation  is  that  preparing  the  AAR  is  a  knowledge  management  challenge. 
The  NTC  provides  a  robust  instrumentation  system  that  offers  a  wealth  of  data.  The  individual 
O/Cs  generates  an  abundance  of  data  through  personal  observations.  Added  to  this  is  the  data 
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obtained  through  coordination  with  other  O/Cs.  Additional  data  is  gained  through  counterpart 
and  training  audience  interactions,  especially  from  hot  washes.  Finally,  critical  data,  including 
operational  models  or  “a-ways”,  is  incorporated  using  sources  outside  the  training  event.  These 
include  previous  training,  authoritative  sources,  such  as  the  CALL,  and  reports  on  or  O/C 
experiences  conducting  in-theater  operations. 

In  the  relatively  short  time  from  the  end  of  a  training  exercise  to  the  start  of  an  AAR,  the 
O/C  has  to  review  all  of  this  data  to  clarify  his  own  understanding  of  what  occurred  and  why.  He 
has  to  filter  and  organize  the  data  and  then  prepare  to  present  it  in  the  AAR.  To  foster  training 
audience  ownership  during  discussions,  the  O/C  must  neither  control  them  completely  nor  force 
a  linear  progression  on  those  discussions.  Therefore,  the  O/C  needs  easy  access  to  all  relevant 
the  data.  In  the  austere  field  environments  where  a  majority  of  AARs  are  conducted,  achieving 
this  is  even  more  difficult. 

The  final  observation  is  the  linkage  between  effective  facilitation  and  basic  leadership. 

FM  6-22,  Army  Leadership  (2006)  identifies  a  leadership  model  with  attributes — what  an  Army 
leader  is,  and  core  leader  competencies — what  an  Army  leader  does.  These  are  similar  to  the 
attributes  and  competencies  which  interviewees  expressed  when  they  described  working  with 
counterparts  and  training  audiences. 


Table  5.  Leadership  Requirements  Model 


Leadership  Requirements  Model 

Attributes- What  an  Army  Leader  is 

Core  Leader  Competencies-What  an  Army  Leader  does 

A  Leader  of  Character 

Army  Values 

Empathy 

Warrior  Ethos 

Leads 

Leads  Others 

Extends  influence  beyond  the  chain  of  command 

Leads  by  example 

Communicates 

A  Leader  with  Presence 

Military  bearing 

Physically  fit 

Composed,  confident 

Resilient 

Develops 

Creates  a  positive  environment 

Prepares  self 

Develops  others 

A  Leader  with  Intellectual  Capacity 

Mental  agility 

Sound  judgment 

Innovation 

Interpersonal  tact 

Domain  knowledge 

Achieves 

Gets  results 

Generally,  the  model  was  supported  by  our  observations.  Observed  behaviors  of 
facilitators  and  group  members  within  AAR  sessions  represented  a  considerable  variety  of 
constructivist  activities.  It  was  clear  to  the  research  team  that  observed  behaviors  reflected  many 
of  the  specific  constructs  within  the  FT AAR  model  (learning,  performance  appraisal,  feedback, 
and  knowledge.)  More  importantly,  it  was  evident  that  the  interpersonal  dynamics  we  observed 
reflected  the  process  oriented  transitional  steps  between  model  components.  For  example,  AAR 
facilitators  spent  considerable  energy  to  specify  goals  and  to  revise  or  update  criteria  as 
discussions  ensued.  We  also  noted  that  AAR  participants  were  generally  engaged  in  mental 
model  assimilation  as  they  incorporated  the  feedback  received  from  other  participants,  command 
personnel,  and  facilitators.  Hypothesized  supporting  constructs  were  useful  for  discriminating 
between  positive  and  negative  behaviors  of  facilitators  and  groups. 
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One  test  of  an  adequate  research  model  is  the  sensitivity  of  its  constructs.  Our  team  was 
very  interested  in  whether  facilitators  and  group  members  would  vary  with  regard  to  the 
constructs  represented  within  the  IT AAR  model.  Our  observations  confirmed  the  power  of  the 
model.  Specifically,  we  observed  in  several  cases  that  facilitators  refined  their  appraisal  content 
in  real  time  based  on  reactions  of  the  Soldiers  and  groups  during  the  AARs.  We  also  noted 
significant  efforts  by  facilitators  to  ensure  that  group  members  achieved  shared  mental  models. 


Table  6.  Cognitive  Research  Areas  That  Contribute  to  Constructs  Within  the  ITAAR  Model 


Research  Area 

Relevant  ITAAR  Construct 

Citation(s) 

Instructional 

Technology 

Feedback,  Knowledge 

Baird,  Holland,  &  Deacon  (1999); 
Briggs  (1977) 

Human  Learning 

Knowledge 

Garvin  (2000);  Bilodeau  (1966); 
Atkinson  &  Shiffrin  (1968) 

Task  Training 

Knowledge,  Feedback, 
Performance  Appraisals 

Holding  (1965);  Patrick  (1992); 
Goldstein  &  Ford  (1992) 

Team 

Performance 

Knowledge,  Feedback, 

Goals 

Minionis,  Zaccaro,  &  Perez  (1995); 
McIntyre  &  Salas  (1995) 

Leadership 

Performance  Appraisals 

Hoyt,  Halverson,  Murphy  & 

Watson  (2003) 

Knowledge 

Management 

Knowledge,  Goals 

Nonaka  &  Takeuchi  (1995);  Joshi, 
K.D.,  Sarker,  S.,  &  Sarker,  S. 

(2007) 

In  cases  where  AARs  were  less  successful,  the  observed  weaknesses  were  directly  related 
to  specific  model  constructs:  flaws  in  shared  knowledge,  failure  to  assimilate  feedback  into 
existing  mental  models,  and  failure  to  engage  in  post-training  discussions.  To  elaborate,  one  of 
the  pervasive  problems  we  noted  was  the  failure  of  AAR  participants  to  spontaneously  engage  in 
dialog.  Such  dialog  is  a  central  part  of  the  ITAAR  model,  linking  knowledge  to  feedback. 
Conversely,  some  of  the  more  effective  AAR  sessions  featured  data  that  were  related  to 
performance  assessment  criteria  and  goal  monitoring. 

Proposed  important  constructs  successfully  discriminated  between  activities  of 
facilitators  (e.g.,  high  on  leadership)  and  group  members  (e.g.,  high  on  team  activity).  During 
the  AAR  sessions  we  observed,  a  critical  aspect  was  the  balance  between  facilitator  and  group 
expression.  The  most  effective  AARs  were  those  where  facilitators  comfortably  relinquished 
control  of  the  session  to  group  members,  so  that  they  could  discover  their  own  insights.  For  their 
own  part,  effective  group  members  were  eager  to  talk  to  each  other,  not  relying  on  the  facilitator 
as  a  go-between  for  shared  information.  The  ITAAR  model  represents  this  difference  well  using 
the  constructs  of  feedback  and  goals.  The  transition  step  from  “Feedback”  to  “Goals”  (Enables 
Self-Goal  Monitoring)  was  intended  for  this  purpose.  Additionally,  the  transition  from  the 
Performance  Appraisals”  to  “Knowledge”  constructs  reflects  the  importance  of  achieving  shared 
mental  models;  such  achievement  is  possible  only  through  mutual  group  member  exchange. 

The  importance  of  proposed  constructs  was  confirmed  based  on  the  observed  behaviors 
of  facilitators  and  group  members.  As  noted  above,  the  exchange  between  facilitators  and  group 
members  was  a  key  focus  for  our  observations.  We  noted  that  there  were  many  occasions  where 
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the  facilitators’  knowledge  of  the  battle  tempo,  dynamics,  and  resources  added  to  the  SA  of  team 
members.  Such  information  exchange  is  reflected  directly  by  the  IT AAR  model  linkage  that 
states,  “Knowledge  of  task  facilitates  goal  setting;  goals  structure  the  knowledge  dispersement 
process.” 

Body  of  Knowledge 

Support  for  training  leaders  on  the  AAR.  Although  one  can  argue  that  the  Army’s  use 
of  AAR  techniques  dates  back  several  decades,  the  seminal  description  of  the  modem  AAR  is 
found  in  the  1993  TC  25-20,  A  Leader’s  Guide  to  After-Action  Reviews.  This  training  circular, 
which  has  never  been  updated,  provides  a  comprehensive  discussion  of  the  processes  and 
techniques  one  should  employ  when  planning,  preparing,  conducting,  and  using  the  results  of  the 
AAR.  This  document  serves  as  the  key  reference  and  touch  point  for  the  various  Army  doctrinal 
manuals  that  address  the  AAR,  such  as,  FM  7-0,  Training  for  Full  Spectrum  Operations  (DA, 
2008)  and  FM  7-1,  Battle  Focused  Training  (DA,  2003).  However,  providing  leaders  and 
trainers  with  doctrinal  manuals  and  training  circulars  that  describe  the  AAR  process  does  not 
teach  leaders  or  trainers  to  effectively  conduct  AARs. 

When  one  explores  the  extent  to  which  the  AAR  is  addressed  as  a  specific  learning 
objective  in  the  Army  professional  education  system,  it  appears  to  be  infrequently  considered. 
This  team  reviewed  the  Programs  of  Instruction  (POI)  for  officers,  NCOs  and  enlisted  personnel 
in  both  the  Field  Artillery  (FA)  and  the  Air  Defense  Artillery  (ADA)  schools.  The  AAR,  as  a 
specified  learning  objective,  could  only  be  found  in  a  small  number  of  officer  and  NCO  courses 
at  the  ADA  school.  In  some  of  these  cases  the  intended  context  for  this  AAR  instruction  was 
strictly  the  conduct  of  an  AAR  of  that  course  and  not  training  how  to  conduct  an  AAR  for  future 
employment  during  unit  training  or  operations.  Instances  of  the  AAR  as  a  learning  objective 
were  not  found  in  any  of  the  FA  POIs. 

Salter  and  Klein  (2007)  discuss  the  inclusion  of  the  AAR  as  a  key  learning  objective  in 
the  Warrior  Leaders  Course  (WLC)  produced  by  the  U.S.  Army  Sergeants  Major  Academy 
(USASMA).  This  is  a  two  hour  block  of  instruction  that  includes  a  45  minute  practical  exercise. 
The  practical  exercise  is  driven  by  a  paper-based  vignette  and  provides  the  opportunity  for  each 
student  to  develop  their  AAR  materials  and  then  participate  in  a  class  discussion  of  the  results. 
This  practical  exercise  does  not  afford  the  time  required  for  each  student  to  practice  the  conduct 
of  their  intended  AAR,  only  to  share  information  on  their  intended  approach.  The  POI  for  the 
WLC  was  recently  updated  but  our  team  could  not  obtain  a  copy  of  the  new  curriculum  so  we  do 
not  know  if  the  two  hour  block  of  instruction  on  AAR  was  included  in  the  updated  POI  or 
modified. 

The  POIs  at  the  O/C  Academies  run  by  the  CTCs  include  a  significant  module  of 
instruction  on  the  AAR.  By  example,  the  O/C  Academy  Training  at  the  JRTC  is  a  three  day 
course  that  reviews  all  of  the  material  an  O/C  needs  to  know  to  work  at  JRTC.  Its  POL 
emphasizes  key  portions  of  the  O/C  Handbook.  Within  this  POI  is  a  block  of  instruction  focused 
on  the  AAR  that  covers  the  key  components  of  TC  25-20  within  the  context  of  training 
operations  at  the  JRTC.  This  includes  AAR  planning,  preparations,  and  formats.  The  class 
provides  the  JRTC  AAR  format  and  example  focus  area  slides  (e.g.,  rehearsals,  priorities  of 
work),  as  well  as  slides  for  specific  focus  items  (e.g.,  command  and  control;  force  protection). 
The  training  repeats  the  admonishment  found  in  most  doctrinal  manuals  not  to  turn  the  AAR  into 
a  lecture  or  critique.  They  note  that  a  critique  provides  only  one  side  of  an  activity  and  tends  to 
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focus  on  what  went  wrong  rather  than  on  the  training  event  as  a  whole.  As  noted  by  all  others 
and  repeated  in  the  O/C  train  up  class,  the  lecture  format  reduces  open  discussion  and  usually 
fosters  neither  team  building  nor  unit  cohesion  (Salter  and  Klein,  2007).  What  the  O/C  Academy 
POIs  do  not  include  are  practical  exercises  that  provide  the  opportunity  for  guided  practice  in  the 
execution  of  the  AAR.  Such  a  practicum  would  serve  to  internalize  the  adult  learning  tactics  and 
techniques  described  in  the  PowerPoint-based  courseware.  According  to  Salter  and  Klein  the 
training  support  materials  employed  in  this  POI  do  not  even  include  exemplar  videos  that 
provide  a  good  view  of  what  “right”  looks  like.  This  lack  of  in-class  opportunity  for  students  to 
practice  and  perfect  their  skills  in  conducting  AARs  means  that  even  O/Cs  at  the  CTCs  must 
learn  via  on  the  job  training.  There  is  no  assurance  that  what  the  O/Cs  learn  from  their  practical 
experience  is  consistent  with  the  core  principles  of  effective  AAR  or  is  consistently  applied 
across  the  team  of  O/Cs. 

The  core  Soldier  and  leader  development  POIs  not  only  rarely  include  instruction  in  the 
fundamentals  of  the  AAR,  they  also  do  not  cover  adult  learning  theory  or  provide  guided 
instruction  and  practical  exercises  where  one  might  develop  AAR  skills.  In  effect,  the  only 
sources  available  for  all  leaders  and  trainers  to  leam  the  core  concepts,  general  methods,  and 
even  tips  to  help  ensure  the  conduct  of  an  effective  AAR  are  Army  doctrine  manuals.  This  fact 
reinforces  the  notion  that  AAR  conduct  is  now  a  skill  that  is  usually  learned  in  the  way  of  an 
apprentice.  Instead  of  leaders  and  trainers  undergoing  a  consistent  AAR  learning  experience  that 
is  reinforced  as  they  progress  up  through  the  levels  of  professional  development  and  schools, 
AAR  skills  come  from  on  the  job  experience  based  on  observing  others  conducting  AARs  and 
from  being  the  recipient  of  “AARs.” 

This  approach,  assuming  leaders  understand  and  can  conduct  effective  AARs,  is  of 
concern  when  one  looks  closely  at  the  doctrine.  FM  7-1,  and  other  doctrinal  manuals,  state 
“Commanders  are  responsible  for  training  O/Cs  to  include  training  on  how  to  conduct  an  AAR.” 
If  the  commanders  have  not  been  formally  trained  on  the  AAR  process  and  the  art  of  conducting 
effective  AARs,  then  we  should  not  assume  that  that  they  can  effectively  train  others  how  to 
conduct  AARs.  Someone  technically  and  tactically  knowledgeable  of  the  appropriate  TTPs  can 
create  and  lead  a  training  critique  using  a  standard  AAR  format.  However,  the  art  of  facilitating 
an  introspective  AAR,  where  a  unit  identifies  areas  for  improvement  and  collaborates  to  develop 
a  plan  for  corrective  action,  requires  guided  training  and  practice;  it  is  a  complex  skill  that  must 
be  learned.  The  techniques  to  conduct  such  an  AAR  and,  even  more  so,  understanding  the 
underlying  cognitive  and  learning  process,  must  be  explicitly  taught  to  Army  leaders. 

Future  research.  In  addition  to  the  practice  recommendations  above,  we  have  identified 
a  number  of  issues  and  areas  where  future  behavioral  and  organizational  research  relative  to  the 
AAR  is  warranted.  Several  of  these  agenda  items  relate  to  the  various  theoretical  foundation 
areas  within  the  IT AAR  model. 

One  topic  that  needs  to  be  investigated  is  the  status  discrepancy  within  groups  attending 
AARs.  There  appear  to  be  frequent  variations  in  organizational  status  among  participants  in 
traditional  industry  AARs.  In  military  training,  the  status  difference  is  clearly  highlighted  by  the 
traditional  chain  of  command  and  indicators  of  rank  or  position  (insignia,  audience  deference). 

In  our  observations,  we  observed  that  status  discrepancies  influenced  the  free  flow  of 
information  and  restricted  audience  remarks;  suggestions  were  frequently  routed  through  the 
battalion  commander  or  the  facilitator  instead  of  being  directed  to  other  participants.  Social 
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identity  is  acknowledged  as  an  influencer  of  learning  (c.f.,  Child  &  Rodrigues,  2006),  therefore  it 
seems  important  to  explore  the  impact  of  social  identity  within  the  feedback  process  of  the  AAR. 

A  second  area  we  recommend  for  future  research  is  investigating  the  impact  of  specific 
technology  tools  on  learning  and  retention.  During  the  AAR  sessions  we  observed,  facilitators 
used  certain  technology  to  support  their  presentation.  However,  it  often  appears  that  the  use  of 
technology  was  tangential  to  the  goal  of  providing  specific,  quality  training  performance 
feedback.  In  some  cases,  video  material  was  shown  to  increase  morale  or  to  impart  humor; 
however,  its  relevance  to  what  was  discussed  during  the  AAR  was  unclear.  In  other  cases, 
facilitators  relied  on  PowerPoint  slides  or  documents  to  portray  “ground  truth”  data.  This  tactic 
generally  seemed  effective,  provided  the  facilitator  could  emphasize  those  points  in  time  where 
the  media  content  was  directly  relevant  to  teaching  points.  Over  the  last  15  years,  researchers 
have  increasingly  debated  the  benefits  of  using  technology  in  the  classroom  (c.f.,  Cradler, 
McNabb,  Freeman,  &  Burchett,  2002).  There  is  usually  limited  time  to  conduct  AARs  (this  was 
frequently  noted  by  trainees  during  our  observations),  therefore  it  is  important  to  optimally  use 
the  time  available.  We  recommend  research  to  investigate  the  best  use  of  technology  during 
AAR  sessions. 

The  third  issue  we  believe  is  appropriate  for  further  involves  comparing  group  dynamics 
during  the  training  session  with  those  apparent  during  the  AAR  session.  There  were  notable 
differences  among  groups  during  the  observed  AARs.  These  differences  were  due  to  numerous 
factors;  however,  a  strong  influence  on  AAR  success  was  the  group’s  cohesiveness.  More 
effective  groups  clearly  possessed  a  collective  passion  and  commitment  to  improvement.  They 
did  not  appear  to  require  significant  direction  or  control  from  the  facilitator  during  the  AAR.  In 
contrast,  some  observed  groups,  which  were  less  cohesive,  required  significant  direction  from 
the  facilitator  during  the  AAR.  These  types  of  group  dynamics  often  reflect  the  presence  of 
group  member  traits  (as  opposed  to  states).  If  this  is  true,  it  may  be  worthwhile  to  determine 
whether  attitudes  and  predispositions  can  be  determined  during  training,  then  used  to  tailor  AAR 
sessions  to  improve  their  outcomes.  This  idea  is  supported  by  researchers,  such  as  Anderson  and 
Martin  (2002),  who  suggest  that  group  communication  skills  may  be  reflected  across  situations. 

Deployment.  We  have  alluded  several  times  in  the  previous  discussions  to  the  need  to 
teach  Army  leaders  (and  other  services  for  that  matter)  the  fundamental  theoretical  principles 
underlying  an  AAR.  An  analogy  is  teacher  education,  in  which  students  in  training  to  be 
teachers  are  taught  not  only  the  subject(s)  they  are  planning  to  teach,  but  also  methods  of 
instruction,  principles  of  learning,  and  concepts  of  cognition.  Because  nearly  all  Army  leaders  in 
the  NCO  and  Officer  ranks  will  sooner  or  later  be  called  upon  to  lead  AARs,  there  is  a  need  for 
them  to  clearly  understand  the  rationale  behind  that  process.  The  IT AAR  model  could  be  used 
as  a  basis  for  teaching  the  key  fundamentals  that  leaders  will  be  expected  to  understand  and 
leverage  in  facilitating  an  AAR.  While  teaching  the  model  per  se  may  be  more  complex  and 
effort-intensive  than  the  value  derived  from  comprehending  it  warrants,  the  key  elements 
captured  in  the  model  (Figure  1)  should  be  used  as  the  basis  for  curriculum  used  in  professional 
education  resident  as  well  as  in  on-line  courses  suitable  for  distance  learning. 
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Recommendations 


Recommendations  for  the  Practice  of  AAR 

The  AARs  we  monitored  were  generally  effective.  Given  the  complexity  of  the  tasks 
performed,  the  heterogeneous  nature  of  the  groups  evaluated,  and  the  compressed  time  available 
for  each  review,  facilitators  were  notably  efficient  as  they  organized  the  AAR  sessions.  Groups, 
too,  were  dedicated  and  responsive.  There  are  several  suggestions  that,  if  implemented,  could 
enhance  the  process  and  produce  better  learning. 

First,  we  observed  that  many  of  the  physical  environments  used  were  not  well  suited  for 
group  interaction.  Group  members  were  often  seated  in  rows,  which  discouraged  them  from 
lengthy  or  face-to-face  interactions.  Repositioning  the  furniture  so  that  group  members  face  each 
other  (such  as  in  a  circle)  could  enhance  interaction,  collaboration,  and  discussion.  Another 
observation  is  that  facilitators  often  used  videos  which  bore  marginal  relevance  to  the  content  of 
the  AARs  themselves.  It  almost  appears  that  they  feel  compelled  to  incorporate  the  videos 
merely  because  the  technology  allows  them  to  do  that.  We  also  recommend  providing  the  AAR 
facilitators  with  Smartboards  to  record  notes,  present  ideas,  or  organize  suggestions  for 
individual  or  collective  improvement.  Doing  so  would  allow  ready  conversion  of  notes  to 
electronic  format,  thereby  ensuring  that  a  record  is  maintained  for  future  use. 

The  presence  of  the  facilitator,  though  beneficial  for  record  keeping  and  direction,  at 
times  appeared  to  stifle  free  exchange  of  information  among  group  members.  It  may  be 
beneficial  to  have  the  facilitator  participate  remotely,  so  that  group  members  perceive  less 
scrutiny  during  social  interchanges. 

Another  suggestion  might  be  to  strategically  locate,  within  the  room,  those  group 
members  who  hold  position  power  (French  &  Raven,  1959),  to  encourage  social  exchanges  to 
flow  through  them.  The  importance  of  physical  seating  arrangement  has  been  noted  in  a  number 
of  therapy  environments  where  exchange  of  information  is  a  critical  part  of  the  learning  and 
growth  process  (Steinzor,  1950). 

ISD  Implications  for  Training/Education  System  Development 

We  learned  that  formal  “Conduct  of  AAR”  training  is  not  included  in  most  Army  POIs 
and  that  this  skill  is  one  most  often  “learned”  via  apprenticeship- style  training.  In  essence,  the 
techniques  employed  are  the  techniques  the  apprentice  has  seen  employed.  Not  always,  but 
often,  AARs  turn  into  unilateral  briefs  or  critiques  governed  by  the  norms  of  the  unit’s 
hierarchical  chain  of  command  rather  than  collaborative  feedback  sessions  that  assist  the  training 
audience  achieve  their  learning  objectives. 

Because  the  theoretical  methodology  for  conducting  an  effective  AAR  often  diverges 
from  standard  military  practice  and  SOPs,  we  recommend  including  that  topic  in  POIs  for  all 
trainers,  facilitators,  observers,  and  Army  leaders  in  general.  Explicitly  pointing  out  how  an 
AAR  is  different  from  other  types  of  briefs,  relating  the  benefits  of  a  successful  AAR,  and 
creating  a  live  practice  environment  where  AAR  skills  can  be  practiced  are  all  topics  that  should 
be  included  in  POI  courseware.  For  a  suggested  curriculum  for  “Conduct  of  AAR”  training,  see 
Appendix  A. 
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Community  of  Interest 

The  results  of  our  research  have  led  us  to  consider  the  wider  theoretical  and  practical 
issues  that  drive  the  military  AAR  process.  We  recognize  that  there  is  a  substantial  community 
of  scientists  and  practitioners  interested  in  the  AAR  process,  and  committed  to  its  effective 
implementation  in  the  field.  Our  research  has  enabled  us  to  communicate  with  such  individuals, 
and  to  take  initial  steps  toward  the  organization  of  a  AAR  community  of  interest  (COI).  That 
community  would  include  members  from  academia,  industry,  and  the  military  who  share 
common  goals:  to  insure  AARs  are  a  key  event  for  enabling  the  training  audience  to  benefit 
from  a  review  of  training;  exploitation  of  appropriate  technology  and  techniques;  enhancements 
to  training  environments  to  better  support  AARs;  and  encouraging  the  commitment  of  trainers  to 
apply  effective  AAR  approaches  both  in  the  field  and  in  classrooms. 

Based  on  these  goals,  we  recommend  the  following  activities  be  initiated  which  will  be 
both  supported  by  and  central  to  instantiating  a  COI: 

1.  There  is  a  need  for  a  funding  program  which  will  support  and  integrate  a 
multidisciplinary  approach  (training,  human  factors,  military,  industrial  psychology, 
education,  others)  to  research.  After  examining  theoretical  and  applied  work 
accomplished  by  a  variety  of  individuals  and  institutions  studying  the  AAR  process,  we 
observed  that  most  work  has  been  accomplished  in  an  ad  hoc  manner  at  the  initiative  of 
organizations  conducting  or  sponsoring  the  research.  It  is  common  for  the  agenda  of 
researchers  to  be  driven  by  the  availability  of  research  funding  and  perceived  needs  of 
sponsoring  organizations.  Therefore,  comprehensive,  multi-year  programs  of  research 
devoted  to  focus  on  areas  such  as  AAR  practice  are  unusual.  We  recommend  that  the 
Army  (or  an  OSD  level  research  organization)  commit  to  a  dedicated  research  program 
on  AAR  theory  and  practice  over  multiple  years.  A  model  for  this  approach  is  the  Virtual 
Technology  and  Environments  (VIRTE)  program  (sponsored  by  the  Office  of  Naval 
Research).  TRADOC  should  be  a  strong  proponent  for  research  to  gain  understanding  of 
the  AAR  and  improving  its  practice.  They  should  take  the  lead  in  promoting  research 
initiatives,  organizing  conferences  and  workshops  for  practitioners  and  researchers  alike, 
and  establishing  forums  for  information  exchange. 

2.  A  collaborative  research  effort  designed  to  compare  and  contrast  AAR  theory  and 
application  across  varied  domain  areas  is  needed  to  replace  what  today  are  largely  efforts 
stove-piped  by  domain  area.  For  example,  there  are  researchers  who  have  devoted  effort 
to  understanding  the  AAR  process  within  the  firefighting  community,  those  who  have 
studied  AAR  use  in  educational  institutions,  those  who  have  studied  AAR  for  particular 
industries,  and  those  who  have  focused  their  research  within  a  military  context. 
However,  we  discovered  that  there  are  significant  lessons  to  learn  from  researchers  who 
are  working  in  other  (seemingly  unrelated)  application  domains.  We  recommend  that 
funding  sponsors  leverage  the  opportunity  for  cross-fertilization  and  establish 
multidisciplinary  grants  that  encourage  the  cross-pollination  of  research  ideas  and 
theories.  Such  an  effort  could  help  to  further  validate  theoretical  foundations  such  as  the 
IT  AAR  model. 

3.  Longitudinal  research  that  investigates  the  AAR  process  from  start  to  finish  should  be 
funded.  During  this  research  we  observed  that  most  intellectual  effort  in  terms  of  the 
AAR  has  concentrated  on  the  conduct  of  formal  AAR  sessions.  However,  the  AAR 
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process  as  practiced  within  the  military  is  much  more  complex.  Military  units  are 
accustomed  to  instituting  informal  hotwashes  or  review  sessions  where  the  focus  is  on 
individual  and  small  group  tactics,  informal  AARs  where  the  focus  is  on  the  performance 
of  a  small  team,  large  AARs  where  integration  of  teams  is  the  focus  (e.g.,  company 
level),  and  very  large  AARs  where  integration,  communication,  and  role  assignment  is  of 
interest  within  large  or  across  commands.  A  comprehensive  research  program  should 
focus  on  not  just  one  or  two  levels  of  AARs,  but  on  the  integration  and  progression  of 
lessons  learned  for  an  entire  training  event  at  multiple  levels  of  command. 

4.  Develop  a  set  of  standards  to  optimize  the  physical  environments  within  which  AARs  are 
implemented.  Because  there  are  a  variety  of  application  domains  for  AARs  there  are 
diverse  types  of  physical  environments  within  which  AARs  are  delivered.  We  observed 
that  the  use  of  technology  and  physical  space  impact  the  degree  with  which  participation 
and  communication  occurs  during  the  AAR.  This  impact  is  often  amplified  within  the 
military  because  of  the  potential  influence  of  the  formal  chain  of  command  relationships 
within  the  training  audience.  Though  Soldiers  are  encouraged  to  speak  freely,  we 
observed  a  free  and  open  exchange  was  often  not  attained.  We  recommend  a  study  be 
conducted  with  an  end  goal  of  recommendations  for  how  to  best  configure  physical 
spaces  to  accommodate  the  AAR  process.  We  recommend  that  these  guidelines  be  based 
upon  principles  and  research  findings  relevant  to  learning  found  in  the  social  and 
educational  literature. 

5.  Training  support  material  should  be  developed  in  the  form  of  a  text  or  reference  book  to 
aid  AAR  practitioners  in  the  field  and  suitable  for  use  within  educational  programs  that 
teach  AAR  theory  and  practice.  Such  a  book  may  be  forthcoming  from  an  independent 
author  or  team,  but  if  not  then  it  should  be  promoted  or  even  sponsored  by  the  Army. 

6.  The  advent  of  the  use  of  game-based,  multi-player  environments  or  virtual  worlds  to 
support  training  is  also  a  key  research  area  that  must  be  addressed  in  the  context  of  how 
to  best  conduct  an  AAR  to  support  this  training  genre.  Game -based  technology  allows 
for  a  distributed  training  audience  and  in-world  interaction  that  correlates  with  live 
training  of  teams  or  groups.  The  issues  that  need  investigation  involve  how  and  where 
one  conducts  an  AAR  of  these  in-world  events.  If  it  is  virtually  (i.e.,  in  world),  which 
appears  to  make  the  most  sense,  then  researchers  need  to  identify  which  of  the  theoretical 
foundations  we  have  captured  in  the  ITAAR  model  apply  and  how  to  best  support  the 
elements  of  that  model  in  a  virtual  setting. 


Summary  and  Conclusions 

The  AAR  has  developed  into  a  key  feature  of  the  Army’s  training  approach.  It  is  being 
conducted  in  various  fashions  throughout  the  Army,  primarily  as  facilitator-led  efforts  to  support 
self  evaluation  within  a  team  or  unit.  Solid  techniques  have  been  developed,  key  research 
conducted,  and  canonical  practice  within  some  organizations  established.  An  opportunity  exists 
to  leverage  what  has  essentially  been  grass  roots  research  and  principles  of  practice  developed 
somewhat  independently  by  a  variety  of  organizations.  We  can  combine  those  efforts  and 
juxtapose  them  with  the  research  results  on  theoretical  foundations  from  this  research.  The 
objective  would  be  to  establish  a  common  basis  for  understanding  what  an  AAR  should 
accomplish  together  with  a  recommended  set  of  principles  to  guide  practice.  That  body  of 
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knowledge  would  support  training  Army  leaders  to  serve  as  facilitators  of  AAR  events  and  guide 
future  development  of  doctrine  and  supporting  AAR  technology  and  systems.  We  recommend 
that  the  IT AAR  model  be  promulgated  together  with  an  explanation  and  include 
recommendations  of  additional  references  in  the  key  theoretical  areas  for  use  by  those  interested 
in  AAR  at  the  conceptual  and  theoretical  levels,  as  well  those  who  do  or  will  teach  AAR 
practitioners.  A  reference  note  or  training  guide  on  the  ITAR  model  with  this  type  of  content 
would  be  appropriate.  There  is  also  a  need  for  a  more  extensive  reference  book  on  AAR  that 
could  be  used  in  future  research  and  to  guide  curriculum  development  (for  teaching  AAR)  but 
such  a  tome  should  be  pursued  by  authors  on  their  own  without  specific  government  funding  and 
marketed  commercially  not  just  to  the  military  but  to  other  government  agencies  and  industry. 
The  AAR  has  been  and  will  continue  to  be  a  key  force  multiplier  in  optimizing  the  value  attained 
in  training.  It  is  a  fiscally  economical  way  to  derive  additional  benefit  from  investment  in 
training  events  and  technology.  In  this  ensuing  era  of  limited  resources  available  to  the  defense 
community  we  need  to  insure  the  use  of  AAR  and  the  support  to  do  it  properly  (Observer 
Controllers,  Senior  Mentors,  replay  and  review  technology,  etc.)  are  not  impacted  by  budgetary 
reductions.  And  the  training  community  and  those  with  an  AAR  Community  of  Interest  need  to 
promote  its  use  even  more  strongly  as  a  cost  effective  way  to  derive  added  value  from  training 
already  funded  and  planned. 
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APPENDIX  A:  Suggested  Curriculum  for  “Conduct  of  AAR”  Training 

Standard  curriculum.  Effective  instruction  should  as  much  as  possible  mimic  the  conditions 
under  which  the  skills  will  be  performed.  A  constructivist  instructional  design  is  appropriate  for 
maximizing  the  replication  of  effectively  facilitating  an  AAR.  A  constructivist  instructional 
design  should  include  the  following: 

E  Anchor  the  learning  activities  by  sharing  the  larger  learning  objective  with  participants. 
This  would  include  showing  examples  of  the  effective  conduct  of  an  AAR  as  well  as 
providing  context  or  background  information  on  why  an  AAR  is  important  in  the  bigger 
picture.  This  might  be  particularly  important  since  the  instructional  methods  and 
outcomes  may  be  foreign  to  many  participants. 

2.  Encourage  ownership  of  the  overall  problem  by  eliciting  learner  accounts  of  previous 
AAR  experiences.  Another  possibility  would  be  to  open  with  an  example  of  a  poorly 
conducted  AAR  and  ask  for  comments.  Selecting  one  of  these  activities  will  help 
learners  adopt  the  problem  as  their  own. 

3.  Build  in  the  course  authentic  tasks  that  provide  the  opportunity  to  conduct  an  AAR. 
Fabricated  or  contrived  tasks  do  not  represent  the  same  type  of  cognitive  challenges 
would  not  require  the  same  learner  buy-in  or  level  of  performance  that  would  be  faced  in 
live  performance. 

4.  Created  multi-faceted  authentic  opportunities  to  conduct  an  AAR.  Rehearsals  that 
replicate  the  conditions  under  which  the  skill  must  be  performed  clearly  increase  the 
transfer  of  the  skills  to  the  workplace.  For  example,  conducting  an  AAR  with  a  group  of 
students  may  be  very  different  than  conducting  and  AAR  with  officers. 

5.  Allow  participants  to  own  the  problem-solving  process.  When  presented  with  a 
challenge  faced  in  a  particular  AAR,  a  specific  approach  or  methodology  should  not  be 
dictated  by  the  curriculum;  learners  should  be  challenged  to  use  their  own  methods  to 
devise  a  solution.  Of  course  the  outcome  of  any  solution  should  be  compared  to  AAR 
best  practices  and  guidelines  before. 

6.  Use  an  open  inquiry  techniques  to  value  as  well  as  challenge  the  learner’s  thinking.  This 
is  both  an  instructional  technique  as  well  as  a  learning  objective.  This  concept  means  that 
learners  would  understand  how  to  craft  questions  that  stay  on  the  leading  edge  of 
participant  thinking,  rather  than  directing  them  to  a  particular  goal. 

7.  Use  collaborative  learning  groups  to  broaden  learning  to  accommodate  the  social 
environment.  For  example,  as  learners  construct  a  mental  model  about  how  to  facilitate 
an  AAR,  discussing  this  model  in  groups  helps  to  crystallize  and  refine  the  level  of 
understanding  and  depth  of  the  model. 

8.  Provide  opportunity  for  reflection  in  the  AAR  curriculum.  Reflection  is  often  overlooked 
as  optional  in  courses  designed  for  a  military  audience.  Research  shows  that  the 
inclusion  of  processing  time  deepens  the  understanding  of  where  new  learning  fits  in  the 
broader  context  of  experience.  Additionally,  it  mirrors  the  recommended  AAR  process. 
When  AAR  facilitators  learn  to  think  reflectively,  AAR  participants  will  be  more  likely 
to  do  so  as  well. 
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Taking  the  above  suggestions  and  formulating  a  curriculum  might  look  something  like  the 
following: 

Terminal  Learning  Objective  (TLO): 

AAR  Facilitators  will  be  able  to  conduct  a  variety  of  facilitation  activities  including  and/or 
specializing  in  the  conduct  of  After  Action  Reviews,  incorporating  the  best  practices  as  defined 
by  appropriate  doctrine  and  principles  of  facilitation;  ultimately  guiding  participants  through  a 
session  that  enhances  their  ability  to  learn  from  experience. 

Enabling  learning  objectives  (ELO): 

Students  will  be  able  to: 

•  Understand  the  potential  value  of  an  AAR 

•  Apply  the  principles  of  effective  AARs  from  the  cognitive  science  domain 

•  Define  facilitation 

•  Understand  how  facilitation  differs  from  other  team  functions,  such  as  leadership 

•  Understand  the  principles  of  facilitation 

•  Effectively  facilitate  a  meeting  of  peers 

•  Create  a  plan  for  data  gathering  and  conduct  of  an  AAR 

•  Prepare  to  facilitate  an  AAR 

•  Select  and  isolate  specific  events  that  are  significant  to  the  AAR  group  in  terms  of 
learning  from  experience 

•  Operate  a  variety  of  technologies  to  expertly  enhance  the  effectiveness  of  AAR 
operation’ 

•  Successfully  rehearse  and  AAR 

•  Display  information  cogently  to  allow  the  AAR  group  to  draw  accurate  conclusions 

•  Conduct  an  effective  AAR 

•  Critique  an  effective  and  ineffective  AAR 

•  Employ  best  practices  to  close  or  wrap  up  an  AAR  session 

Exemplar  Curriculum: 

I.  Introduction 

a.  Video  of  an  expertly  conducted  AAR 

b.  Discussion  of  participant  experience  with  AARs 

c.  Provide  an  overview  of  the  AAR  process  and  goals  -  with  more  detail  to  be 
delivered  in  subsequent  lessons 

d.  Formative  Activity:  on  a  class  wiki,  students  will  create  a  page  where  they  will 
begin  a  discussion  regarding  the  definition  of  inquiry  learning  and  how  it  applies 
to  conducting  an  AAR. 

e.  Formative  Activity:  students  with  create  individual  blogs  regarding  their 
experiences  with  AARs.  The  first  entry  will  be  a  summary  of  the  above 
discussion. 

f.  Presentation  of  terminal  learning  objectives  and  purpose/importance  of  the  AAR 
process 
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II.  Theoretical  context  for  AAR  organizing  principles 

a.  Formative  Activity:  from  a  pre-selected  book  list,  each  student  will  select  a  book 
to  read  and  review,  outlining  the  3  or  4  key  points  of  the  book.  The  review  will 
be  posted  on  the  class  wiki. 

b.  Behavioral  driven  AAR  research  of  best  practices 

i.  Long  term  memory  storage 

ii.  Situational  Awareness 

iii.  Team  dynamics 

iv.  Communication 

v.  Goal  setting  and  communication 

c.  Design  related  best  practices:  development  of  task  mastery 

d.  The  nature  and  effectiveness  of  feedback  mechanisms 

e.  Formative  Activity  -  students  will  conduct  their  own  AAR  research  and  create  a 
blog  entry  that  adds  more  detail/depth  to  at  least  one  of  the  theoretical  contexts 
presented  above 

III.  Facilitation;  the  context  for  Conducting  an  AAR 

a.  Formative  Activity:  Assign  the  class  the  task  of  organizing  a  panel  of  experts  to 
speak  about  the  AAR  process,  the  instructor  should  expertly  facilitate  the  panel 
of  experts 

b.  Debrief  the  task  by  conducting  a  formal  AAR  on  the  organization  of  the  event 

c.  Conduct  a  “meta”  AAR  (or  an  AAR  on  the  AAR) 

d.  Formative  Activity:  students  will  create  a  blog  post  on  what  they  noticed  about 
the  role  of  facilitator 

e.  Define  facilitation  and  derive  the  elements  of  facilitation  from  the  previous 
activity 

f.  Define  a  broad  range  of  possible  methodologies  and  processes  related  to 
facilitation,  noting  when  each  is  likely  to  be  most  useful 

g.  Present  background  and  research  on  facilitation  and  its  benefits 

IV.  Planning  for  an  AAR 

a.  Present  the  goals  of  the  planning  phase  for  an  AAR 

b.  15  minute  multimedia  presentation  on  planning  an  AAR,  highlighting  the  exercise 
with  video  clips  of  a  planning  session 

c.  Formative  Activity:  Introduce  a  field  trip  to  witness  a  military  exercise/activity 
and  view  an  AAR.  Before  the  trip,  but  with  details  of  the  exercise,  conduct  a 
planning  session  for  the  AAR  of  the  exercise.  [Compare  the  classroom  plan  to  the 
plan  used  by  the  AAR  facilitator] 

d.  Formative  Activity:  Ask  students  to  work  in  teams  of  two  to  create  a  planning 
template  and  post  on  the  class  wiki 

e.  Summative  activity:  Ask  students  to  find  an  event  in  the  future  in  their  place  of 
work  for  which  they  can  conduct  an  AAR  and  create  a  plan. 

V.  Preparing  for  an  AAR 

a.  Present  the  goals  of  the  prepare  phase  of  the  AAR 
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b.  15  minute  multimedia  presentation  on  preparing  for  an  AAR  -  emphasizing  the 
art  of  formulating  questions 

c.  Formative  Activity:  Add  more  detail  about  the  AAR  trip  and  outline  “Prepare” 
tasks  in  relationship  to  the  actual  AAR  event.  [Attend  the  event]  Compare  the 
classroom  preparation  tasks  to  those  of  the  expert  AAR  facilitator.  Ask  students 
to  record  the  questions  asked  and  facilitation  methods  used  during  the  AAR 

d.  Summative  activity:  Create  an  outline  for  preparing  for  the  work  AAR. 

VI.  AAR  Rehearsal 

a.  Present  the  goals  of  the  rehearsal  phase  of  AARs 

b.  Introduce  and  use  the  technologies  associated  with  AARs 

c.  10  minute  presentation  on  the  elements  of  an  AAR  rehearsal 

d.  Formative  Activity:  as  a  class  develop  a  list  of  possible  unexpected  events  and 
ask  the  students  to  devise  strategies  to  overcome  or  work  around  the  interruptions 

e.  Summative  activity:  Ask  students  to  rehearse  their  upcoming  AAR  in  teams  of  4. 
Allow  students  to  coach  on-another  during  the  rehearsal  Complete  several 
iterations,  with  a  different  interruption  each  time. 

VII.  Conducting  an  AAR 

a.  Deliver  a  15  minute  presentation  on  preparing  for  an  AAR 

b.  Show  a  video  of  a  poorly  conducted  AAR  and  an  expertly  conducted  AAR 

c.  Compare  and  contrast  the  field  trip  experience  what  has  been  learned  in  the 
classroom  to  date. 

d.  Formative  Activity:  ask  students  to 

e.  Develop  a  list  of  best  practices  during  class  discussion  and  post  the  results  on  the 
wiki 

VIII.  Closing  an  AAR 

a.  15  minute  presentation  on  wrapping  up  an  AAR 

b.  Formative  Activity:  Have  students  practice  wrapping  up  an  AAR  by  showing  a 
video  of  an  expert  AAR,  but  stop  just  before  the  closing  and  have  students  supply 
the  ending. 

c.  Formative  Activity:  Students  will  create  a  blog  post  regarding  how  they  expect 
to  close  their  upcoming  AAR  debut. 

d.  Summative  activity:  Ask  students  to  execute  their  work  AAR  and  have  the 
participants  complete  a  survey  upon  completion.  Analyze  the  results  of  the 
survey 
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APPENDIX  B:  AARs  in  Practice 


Planning 

Facilitators  review  unit  training  objectives  and  other  pertinent  information  such  as  past 
unit  training  assessments  and  performance,  unit  composition  including  equipment,  and  personnel 
shortages.  Facilitators  review  the  training  event  schedule.  They  identify  time  and  other  resource 
availabilities  as  well  as  identifying  the  best  time  to  conduct  formal  and  informal  AARs.  AARs 
are  conducted  when  there  is  time  available  within  the  training  event  to  prepare  and  conduct  the 
AAR,  when  the  break  from  training  will  not  significantly  disrupt  training,  and  when  there  are 
sufficient  training  results.  Formal  AARs  consume  larger  amounts  of  preparation  and  execution 
time  and  are  significantly  more  disruptive.  Informal  AARs  consume  less  time.  Other  types  of 
performance  reviews,  i.e.  During  Action  Reviews  (DARs),  also  called  hot  washes,  Green  Book 
AARs,  hood  top  AARs,  etc.,  require  the  least  amount  of  time  and  are  least  disruptive. 

Facilitators  use  DARs  for  reviewing  performance  between  informal  or  formal  AARs  and  to  help 
build  towards  the  eventual  AAR. 

Facilitators  review  the  planned  training  event  scenario.  They  wargame  likely  friendly, 
neutral,  and  enemy  courses  of  action  (COAs)  and  work  with  scenario  writers  and  opposing  force 
(OPFOR)  planners/leaders,  as  they  are  available.  The  wargame  includes  the  traditional  construct 
of  actions,  re-actions,  and  counter-actions.  Facilitators  consider  the  ability  of  the  simulation 
system  to  model  or  replicate  possible  actions.  The  wargame  should  generate  an  initial  collection 
plan. 

Facilitators  review  doctrine;  tactics,  techniques,  and  procedures  (TTP);  lessons  learned; 
and  other  written  material  pertinent  to  unit  training  objectives  and  projected  unit  missions.  They 
refresh  their  understanding  of  sub-tasks,  standards,  and  measures  of  performance  and 
effectiveness.  Unit  training  objectives  drive  the  focus  of  this  pre-execution  review.  Training 
objectives  focused  on  collective  physical-domain  competencies,  i.e.  war  fighting  functions  and 
tactical  tasks,  should  drive  review  of  tactical  doctrine.  Training  objectives  focused  on  collective 
cognitive-domain  competencies,  i.e.  Team  Dimensional  Training,  Think  Like  a  Leader,  Combat 
Profiling/Hunter,  etc;  should  drive  review  of  available  source  documentation.  This  review 
should  result  in  assemblage  and/or  development  of  data  collection  forms  and  other 
capture/organizational  measures  such  as  checklists. 

Facilitators  identify  existing,  pertinent  presentation  material  such  as  PowerPoint  slides. 
They  review  how  this  or  other  training  units  previously  conducted  the  mission  and  collect 
material  from  previous  missions  for  potential  presentation.  Facilitators  store  all  presentation 
material  in  a  place  and  manner  that  allows  for  quick  access  during  AAR  preparation  and  at  the 
AAR  location. 

From  wargaming,  facilitators  identify  expected  key  events.  Key  events  are  those  training 
unit  actions  that  lead  directly  to  unit  success  or  failure.  Unit  actions  may  include  both  collective 
(physical)  and  leader  (cognitive)  tasks,  i.e.  decisions.  Emphasis  on  collective  vs  leader  tasks, 
including  emphasis  for  targeting  sensors  (to  be  discussed  further  below),  should  be  based  upon 
whether  unit  training  objectives  are  focused  on  tactical  or  cognitive  tasks.  Key  events  are 
associated  with  unit  interactions  with  other  scenario  elements  such  as  OPFOR  or  neutrals.  The 
unit’s  military  decision  making  and/or  troop  leading  procedures  (MDMP  and/or  TLP)  and  the 
operation’s  decisive  point  are  natural  key  events.  Other  key  events  might  include  transition 
between  phases,  initial  contact,  actions  on  the  objective,  potential  medevacs,  etc.  Likely  key 
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events  become  Named  Areas  of  Interest  (NAIs).  As  per  doctrine,  NAIs  are  phased  or  have 
opening  and  closing  conditions,  such  as  time  or  unit  location.  NAIs  associated  with  physical 
tasks  can  be  considered  Targeted  Areas  of  Interest  (TAIs)  while  those  associated  with  cognitive 
tasks  can  be  considered  Decision  Points  (DPs). 

Facilitators  develop  a  data  collection  plan;  identifying  resources  (sensors)  available  for 
data  collection.  Sensors  include  the  facilitator,  other  facilitators  (such  as  O/Cs  or  O/Ts),  exercise 
control  (EXCON)  personnel,  and  training  systems  instrumentation.  Sensors  may  also  include 
training  unit  personnel  and  unit  instrumentation  which  collect  data  as  part  of  their  normal 
operations  and  will  be  sources  of  information,  either  prior  to  the  AAR  through  conversation  or  in 
the  AAR  as  a  result  of  questioning. 

Facilitators  plan  the  positioning  of  sensors  within  the  training  space  and  time  to  collect 
desired  data,  with  emphasis  on  open  NAIs  but  also  following  the  unit  between  NAIs.  Sensors 
should  form  a  network  linking  back  to  the  facilitator.  The  facilitator  controls  location  and 
targeting  of  the  network;  targeting  includes  focusing  on  key  locations,  events,  decisions, 
information  flows,  etc.  Sensors  within  the  network  should  provide  redundant  coverage, 
especially  as  the  targeted  information  becomes  more  fleeting  and  critical.  The  facilitator  orients 
the  sensor  network  towards  the  above  identified  key  events,  i.e.  NAIs.  The  facilitator  also 
identifies  gaps  in  the  network  coverage  for  remediation.  Finally,  the  facilitator  plans  sensor 
locations  and  targeting  so  as  to  minimize  any  disruption  to  training  unit  performance.  The 
primary  sensor  is  the  facilitator  him/herself  and  he/she  should  plan,  if  possible,  on  personally 
observing  the  most  critical  information  for  each  key  event/NAI.  Ideally,  the  sensor  network 
should  be  invisible  to  the  training  unit.  This  includes  the  positioning  of  the  facilitator 
him/herself. 

Facilitators  plan  the  method  they  will  use  to  monitor,  with  assistance,  the  sensor  network 
to  perceive  what  the  training  unit  is  doing  and,  more  importantly,  why  they  are  doing  it.  The 
facilitator  also  plans  methods  to  manage  the  data/information  presented  through  the  sensor 
network.  This  plan  must  include  methods  to  filter  and  perceive  data  in  order  to  identify  critical 
information.  Critical  information  should  be  identified  in  real  time.  However,  this  is  not  always 
possible.  At  times,  data  will  likely  overwhelm  the  ability  of  the  facilitator  and  his  supporters  to 
properly  manage  it.  The  facilitator  should  anticipate  when  this  is  likely  to  occur  and  create  the 
ability  to  designate  pointers  to  important  data  to  help  processing  during  the  AAR  preparation 
phase;  an  example  being  use  of  bookmarking.  Wargaming  will  assist  in  identifying  when  data 
will  overwhelm  monitoring  and  managing  capability. 

Facilitators  identify  desired  AAR  locations  and  coordinate  participation  by  appropriate 
support  personnel.  An  initial  priority  of  the  AAR  should  be  the  identification  and  collective 
visualization  of  what  actually  occurred  within  the  training  event.  Thus,  AAR  locations  should  be 
selected  such  that  they  best  support  visualization  of  the  training  space.  The  next  priority  is  the 
discussion  of  the  reasons  for  the  demonstrated  performance.  The  location  should  support  this 
priority  by  providing  space  where  the  training  unit  can  temporarily  escape  the  stresses  of  training 
and  focus  on  self-assessment.  AAR  locations  support  this  requirement  by,  as  much  as  possible, 
providing  a  level  of  comfort  and  by  controlling  access  and  otherwise  minimizing  if  not 
eliminating  external  distractions,  such  as  noise,  outside  discussion,  etc. 

The  facilitator  coordinates  this  plan  with  other  facilitators,  EXCON,  and  other  personnel 
in  position  to  support  data  collection,  transmission,  and  management.  Coordination  includes 
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rehearsing  the  plan,  with  an  emphasis  on  rehearsing  the  data  collection  effort,  especially  the 
positioning  and  targeting  of  the  sensor  network  and  its  movement  based  upon  opening  and 
closing  of  NAIs. 

Training  Unit  Link-Up 

Link-up  is  the  initial  meeting  between  the  facilitator  and  the  training  unit  prior  to  the  start 
of  training.  Link-up  is  the  last  opportunity,  prior  to  execution  of  training,  for  the  facilitator  to 
observe  the  training  unit,  determine  last  minute  training  unit  status,  and  determine  any 
refinement  of  training  objectives.  Link-up  is  also  the  primary  opportunity  for  the  facilitator  to 
establish  an  appropriate  relationship  with  the  training  unit  and,  especially,  with  the  training  unit 
leadership  or  “counterpart”.  Finally,  link-up  is  the  first  and  primary  opportunity  for  the 
facilitator  to  establish  a  positive  impression  upon  the  training  unit,  including  personally 
modeling  expected  unit  personnel  behaviors. 

Ultimately,  the  objective  of  the  training  event  is  the  transference  of  training  from  the 
event  to  day-to-day  unit  performance.  This  transference  occurs  best  when  the  training  unit 
internalizes  the  training  lessons.  This  implies  ownership  of  those  lessons  by  the  training  unit. 
Thus,  the  relationship  between  the  facilitator  and  his/her  counterpart  should  be  a  partnership, 
where  the  training  is  ultimately  owned  by  the  training  unit  leadership  and  the  facilitator  is  seen  as 
an  assist  to  that  leadership.  In  link-up,  the  facilitator  has  the  ability  to  observe  the  training  unit 
and  assess  the  unit’s  ability  to  take  ownership  of  the  training.  Subsequent  facilitator  actions, 
including  level  of  directive  vs.  participative  coaching,  will  be  dependent  upon  this  initial  and 
ongoing  assessment.  The  facilitator  should  also  be  looking  for  which  individuals  might  be  more 
conducive  to  taking  ownership  of  training  for  later  utilization  by  the  facilitator  as  he/she 
facilitates  learning.  Conducive  individuals  may  not  necessarily  be  unit  leadership.  The  link-up 
is  also  the  opportunity  by  the  facilitator  to  establish  his/her  legitimacy  with  the  training  unit. 

This  legitimacy  is  necessary  to  establish  the  relationship  and  provide  legitimacy  later  within  the 
AAR.  It  is  best  established  subtly  by  professional  behavior  and  by  passive  demonstration  of 
experience. 

Note  that  link-up  is  more  critical  when  the  facilitator  is  not  part  of  the  training  unit’s 
existing  chain  of  command. 

Data  Collection  and  Management 

As  training  occurs,  the  facilitator  executes  his/her  collection  plan,  adjusted  to  account  for 
unit,  OPFOR,  and  neutral  changes.  The  facilitator  pays  especially  close  attention  to  unit  tactical 
planning  and  preparation,  i.e.  MDMP/TLP  because  these  serve  as  key  events  and  because 
MDMP/TLP  will  indicate  changes  to  subsequent  key  events.  The  facilitator  stays  in  constant 
communication  with  the  sensor  network  including  other  facilitators,  EXCON,  etc;  relaying  what 
the  unit  is  doing,  identifying  variances  from  expected  actions,  making  changes  to  the  collection 
plan,  making  timely  decisions  as  to  network  repositioning/retargeting,  and  making  or 
recommending  changes  in  the  scenario  execution  to  ensure  training  objectives  are  met,  assuming 
the  facilitator  has  this  responsibility  and  authority.  Monitoring  includes  development  and 
maintenance  of  Level  3  Situational  Awareness,  or  such  awareness  that  the  facilitator  can  predict 
future  events  within  the  training  event. 

There  may  be  opportunity  during  the  scenario  to  conduct  During  Action  Reviews 
(DARs).  DARs  are  conducted  as  the  situation  permits.  Their  benefit  is  the  immediacy  and 
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frequency  of  review.  They  also  provide  an  opportunity  for  the  facilitator  to  interact  directly  and 
closely  with  unit  leadership  and/or  other  training  unit  personnel.  DARs  are  not  substitutes  for 
AARs,  but  augmentation  to  the  AAR  process  in  particular  and  training  in  general.  DARs  should 
not  only  be  used  for  review  but  also  to  exchange  information  in  preparation  for  an  eventual 
AAR. 

AAR  Preparation 

AAR  preparation  is  a  constant  process  but  formally  begins  at  Change  of  Mission  (CoM) 
or  ENDEX.  Good  preparation  is  critical  to  a  successful  AAR.  During  preparation,  the  facilitator 
completes  data  gathering  and  conversion  of  data  into  information.  He/she  then  prioritizes  and 
organizes  the  information  gathered  in  order  to  best  support  unit  training  objectives.  Finally,  the 
facilitator  familiarizes  him/herself  and  any  AAR  support  personnel  with  the  information 
organization  so  that  during  the  AAR,  information  is  readably  available  to  support  the  ultimate 
direction  of  the  AAR. 

Upon  CoM,  the  facilitator  begins  to  exchange  information  with  all  of  the  elements  of  the 
sensor  network,  including  other  facilitators,  EXCON,  OPFOR  and  role  players,  and  support 
personnel.  Use  of  previously  developed  collection/management  tools  is  essential  to  quickly 
organize  and  filter  raw  data.  Also  important  are  unit  training  objectives  and  any  higher  echelon 
facilitator  intent  or  information  requirements.  The  ultimate  purpose  of  this  step  is  to  establish 
SA  of  what  occurred  and  why.  The  facilitator  should  scan  the  data  flow  to  start  identifying 
trends,  especially  trends  associated  with  previously  identified  key  events.  These  trends  become 
an  initial  list  of  AAR  focus  areas,  or  topics  for  potential  discussion  within  the  AAR.  Note  that 
the  facilitator  should  not  expect  that  every  focus  area  will  be  discussed  in  the  AAR.  The  purpose 
of  focus  areas  is  to  ensure  the  facilitator  has  correct  information  to  support  the  eventual  direction 
of  the  AAR. 

Once  the  facilitator  has  sufficient  SA,  he/she  should  meet  with  their  unit  leader 
counterpart.  One  purpose  of  this  meeting  is  to  exchange  facilitator  and  unit  leader  SA.  This 
serves  as  part  of  the  facilitator’s  data  collection  and  management.  The  primary  purpose, 
however,  is  to  compare  notes  on  that  SA  for  the  focusing  of  the  eventual  AAR.  Again, 
ownership  by  training  unit  personnel  and,  especially,  by  training  unit  leadership  of  the  training 
lessons  is  critical  to  transfer  of  training  from  the  training  event  to  permanent  changes  in  unit 
performance.  The  end  result  of  this  exchange  is  a  finalized  and  agreed  upon  list  of  prioritized 
AAR  focus  areas.  Again,  not  all  focus  areas  will  be  covered  in  the  AAR.  Also  during  this 
meeting,  the  facilitator  and  unit  leadership  identify  who  from  the  training  unit  will  attend  the 
AAR,  where  the  AAR  will  be,  when  it  will  start  and  how  long  it  will  last,  and  any  other  special 
instructions  such  as  uniform  or  specific  information  or  items  the  facilitator  and  unit  leadership 
want  the  unit  to  have  present  at  the  AAR  and  who  will  ensure  it  is  present. 

The  next  facilitator  step  is  to  organize  available  information  for  quick  recall  by  the 
facilitator  during  the  AAR.  The  facilitator  works  with  any  support  personnel  to  ensure  they  are 
well  familiar  with  the  information  available,  its  relationship  to  identified  focus  areas,  and  the 
quickest  means  to  call  up  and  visualize  the  information.  This  includes  presence  at  the  AAR  of  an 
OPFOR  representative  and  critical  role  players,  as  required.  Additionally,  the  facilitator  should 
start  to  identify  open-ended  questions  he/she  will  use  in  the  AAR  to  support  the  various  focus 
areas.  Open-ended  questions  are  questions  requiring  an  answer  other  than  yes -no.  The 
facilitator  can  use  questions  developed  during  preparation  or  bring  particularly  effective 
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questions  used  in  prior  AARs.  Identifying  open-ended  questions  includes  an  initial  decision  of 
to  whom  to  ask  the  question.  Open-ended  questions  can  be  asked  of  individuals  or  the  group  in 
general.  The  facilitator  identifies  individual  answerers  based  upon  an  assessment  of  who  will 
provide  an  answer  that  will  best  foster  group  review,  not  necessarily  who  is  most  likely  to 
provide  the  best  answer.  To  support  control  over  AAR  content,  at  all  times  when  the  facilitator 
asks  a  questions,  he/she  should  have  a  level  of  confidence  in  the  anticipated  answer,  i.e.  he/she 
should  generally  not  answer  a  question  for  which  they  do  not  already  know  the  answer. 

Important  at  this  final  stage  of  preparation  is  an  AAR  rehearsal.  The  purpose  of  the 
rehearsal  is  to  ensure  the  smooth  execution  of  the  AAR.  This  starts  with  inspection  of  the  AAR 
location  to  ensure  it  meets  minimum  standards:  it  supports  visualization  of  unit  performance,  it 
is  large  enough  for  the  AAR  audience  and  AAR  facilitators  including  potential  ingress  and 
egress  by  OPFOR  representative  and  other  special  briefers;  is  configurable  to  support 
visualization  of  presented  material  AND  to  foster  face-to-face  exchange  between  AAR 
participants;  is  relatively  comfortable;  hinders  access  by  non-unit  members;  and  insulates  the 
AAR  from  outside  disruptions.  The  facilitator  ensures  that  all  physical  presentation  material  is 
present  and  that  the  location  includes  blank  paper  charts  for  note  taking  by  the  facilitator. 
Ensuring  smooth  execution  also  includes  coordination  of  communications  between  the  facilitator 
and  AAR  support  personnel.  It  is  critical  that  the  facilitator  knows  exactly  what  the  OPFOR 
representative  and  any  key  role  players  will  say  and  that  should  be  rehearsed.  The  OPFOR 
representative  and  other  key  roles  players  are  generally  not  part  of  the  AAR  discussion  and 
should  depart  the  AAR  after  their  presentation.  Finally,  the  facilitator  and  support  personnel 
should  rehearse  the  quick  recall  and  presentation  of  material  anticipating  the  flexible  flow  of  the 
AAR. 

AAR  Execution 

The  AAR  execution  starts  with  the  arrival  of  the  training  unit  participants.  Unit 
leadership  should  establish  accountability  of  personnel  and  report  that  all  are  present  or  explain 
why  individuals  are  not.  The  facilitator  and  unit  leadership  should  immediately  establish  a 
positive  and  professional  learning  atmosphere.  Participants  should  remove  helmets,  MOFFE, 
and  other  burdensome  equipment  and  sit  down  to  become  comfortable.  Participants  should  be 
expected  to  have  note  taking  material. 

The  facilitator  introduces  him/herself  and  welcomes  participants  to  the  AAR.  He/she 
then  provides  the  rules  of  the  AAR,  to  include: 

•  AAR  is  not  a  critique.  No  one,  regardless  of  rank,  position,  or  strength  of  personality,  has 
all  of  the  information  or  answers.  Goal  is  to  learn  from  each  other. 

•  An  AAR  does  not  grade  success  or  failure.  There  are  always  weaknesses  to  improve  and 
strengths  to  sustain. 

•  Focus  of  the  AAR  is  going  to  be  centered  on  key  events  and  themes. 

•  Facilitator  guides  the  discussion;  participants  provide  input  from  their  point  of 
view/perspective. 

•  Minimal  discussion  on  merits/faults  of  the  simulation. 

A  common  technique  within  AARs  is  to  provide  a  “two-minute  tape”,  or  a  short,  narrated 
video  or  other  presentation  of  the  scenario  execution.  This  serves  to  establish  an  initial  “ground 
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truth”  and  helps  shift  future  discussion  away  from  what  happened  to  why  it  happened.  If  used, 
the  two-minute  tape  must  be  non-judgmental,  discussing  only  what  happened  without  implying 
correctness. 

The  facilitator  expedites  presentation  of  friendly  and  OPFOR  missions  and  intents, 
starting  two  echelons  up.  These  statements  are  usually  written  on  charts  or  slides  and  a 
technique  is  to  have  the  unit  leadership  or  other  member  read  the  friendly  missions  and  intents. 
An  OPFOR  representative  briefs  the  OPFOR  mission,  intent,  and  succinct  plan.  The 
representative  leaves  immediately  afterwards.  If  there  is  no  representative,  the  facilitator  briefs 
the  OPFOR  information. 

The  facilitator  then  begins  the  discussion  based  upon  the  prioritized  list  of  focus  areas. 
The  discussion  should  be  a  participative  and  flexible  review,  with  the  greatest  emphasis  on 
training  audience  acceptance  of  training  lessons  and  eventual  training  transfer.  There  are  several 
facilitator  behaviors  that  will  support  the  participative  review: 

•  The  facilitator  is  a  participant  in  the  AAR  and  must  demonstrate  desired  AAR  participant 
behaviors.  He/she  must  be  professional  in  appearance  and  demeanor.  They  must  at  all 
times  be  positive.  Their  statements  are  non-attribution.  They  avoid  assigning  blame, 
especially  to  individuals.  Rather,  all  performance  issues  are  approached  as  unit  issues 
and  emphasis  is  on  why  they  occurred  and  how  to  rectify  them  through  changes  to  unit 
behaviors.  The  facilitator  is  polite  yet  firm.  When  not  speaking,  they  are  intentionally 
taking  notes  to  demonstrate  the  behaviors  they  expect  from  other  participants,  to  include 
active  listening  in  general  and  note  taking  in  particular.  The  facilitator  takes  notes  on 
available  chart  paper. 

•  The  facilitator  is  constantly  scanning  the  entire  audience  to  gauge  the  AAR’s 
effectiveness.  He/she  is  looking  primarily  for  engagement  in  the  review  process. 
Engagement  is  indicated  in  verbal  participation/contribution  and  in  such  non-verbal 
communication  as  attention,  attitude,  and  individual  note  taking.  The  facilitator  should 
be  cognizant  that  although  contribution  is  expected  from  all  participants,  individual 
participant  learning  styles  will  vary  and  there  may  be  individuals  who  best  learn  through 
passive  observation.  The  facilitator  should  work  to  cause  universal  participation  though 
they  should  remember  that,  although  participation  is  important,  the  most  important 
outcome  of  the  AAR  is  ownership  by  the  unit  of  training  lessons  and  subsequent  training 
transfer.  Too  much  pursuit  of  participation  may  have  an  opportunity  cost  in  less  focus  on 
lessons  and  transfer  of  ownership.  This  is  the  art  of  the  AAR.  Generally  speaking,  there 
are  within  the  AAR  audience  individuals  more  critical  to  successful  transfer,  such  as  unit 
leadership,  and  these  individuals’  participation  might  be  more  critical  than  the 
participation  of  other  individuals. 

•  The  primary  means  for  fostering  group  participation  and  interaction  is  through  the  use  of 
open-ended  questions.  These  are  questions  that  require  something  other  than  a  yes-no 
answer.  The  facilitator  may  ask  questions  of  individuals  or  the  group.  They  make  sure 
all  members  of  the  training  audience  participate  fully  in  the  AAR  by  consistently  asking 
open-ended  questions  and  taking  care  to  solicit  responses  from  all,  not  just  those  that  are 
outspoken.  The  primary  purpose  of  asking  an  open-ended  question  is  to  foster  group 
participation  and  review,  not  automatically  to  elicit  the  best,  most  accurate,  or  most 
complete  answer.  In  addition  to  open-ended  questions,  the  facilitator  should  be  prepared 
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to  ask  follow-up  questions.  These  are  “so  what”  questions  purposed  to  elicit  a  deeper 
understanding  of  what  happened  and  why.  They  can  also  be  used  to  foster  group 
discussion  by  redirecting  statements  or  questions  to  within  the  group.  Follow-on 
questions  might  be  standard,  such  as: 

o  Why  do  you  think  that  way? 

o  What  do  you  think  of  what  was  just  said? 

o  How  could  it  have  been  done  differently? 

•  The  facilitator  should  use  records  (video,  audio,  statistics,  etc)  of  unit  performance 
primarily  to  help  establish  ground  truth,  or  what  actually  occurred  in  the  training  space. 
Use  of  records  is  dangerous  as  it  can  be  interpreted  by  individuals  as  “gotcha”  material, 
or  an  effort  to  attribute  blame. 

•  The  facilitator  assists  in  training  transfer  by  making  sure  the  training  is  perceived  as 
relevant  to  the  training  audience.  To  do  this,  he/she  can  relate  the  training  to  operational 
requirements  and/or  demonstrate  the  legitimacy  of  the  training  in  addressing  those 
requirements. 

•  The  facilitator  should  demonstrate  support  for  the  training  audience’s  experiences.  For 
example,  recognize  and  positively  reinforce  the  training  audience’s  correct  actions  in  the 
training  event.  It  is  also  important  to  mentor  the  training  audience.  Therefore,  the 
facilitator  should  expedite  a  full  explanation  of  the  rationale  behind  correct  actions  or 
guide  the  training  unit  towards  a  well-informed,  accurate  assessment  of  the  training  event 
and  performance,  as  needed. 

•  The  facilitator  is  an  expert  information  source.  Information  brought  by  the  facilitator 
includes  doctrinal  norms  from  published  doctrine  and  TTP.  Information  also  includes 
examples  of  other  methods,  such  as  how  other  units  executed  in  training  and,  more 
importantly,  how  these  tasks  are  being  done  in  active  theaters. 

•  The  facilitator  acts  to  prevent  non-productive  interaction.  This  may  include  individual 
blaming,  or  blaming  performance  on  causes  external  to  training  unit  control,  such  as 
technical  limitations,  equipment  or  personnel  shortages,  or  scenario  faults.  The  facilitator 
prevents  non-productive  interaction  by  emphasizing  “why”  and  avoiding  “who”.  They 
use  plural  nouns  such  as  “we”  and  “us”  in  their  conversation.  Most  importantly,  they 
reinforce  that  good  organizations  look  internally  to  fix  problems  rather  than  externally  to 
blame  problems. 

•  Facilitators  should  be  open  and  up  front  if/when  they  identify  their  own  information  gaps. 
They  avoid  discussing  subjects  of  which  they  have  no  or  insufficient  knowledge.  If  these 
subjects  arise,  they  refer  back  to  the  unit  participants  to  provide  information.  They  note 
these  personal  information  gaps  and  act  after  the  AAR  to  fill  them. 

AARs  are  events  within  a  training  process  or  continuum.  The  facilitator  incorporates 
previous  events,  such  as  earlier  AARs  or  DARs.  This  incorporation  includes  identification  of 
trends  and  discussion  of  unit  responses  to  these  trends. 

Ownership  of  the  AAR  by  the  unit  participants  is  critical.  This  means  that  the  training 
audience  must  have  some  control  over  the  actual  direction  of  the  AAR.  The  facilitator  should 
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accept  this  and  is  prepared  for  it  by  being  very  familiar  with  the  information  on  unit  performance 
and  being  able  to  work  with  support  personnel  to  call  up  information  as  required.  This  requires 
non-linear  access  to  information.  The  facilitator  is  also  prepared  for  this  flexibility  through  prior 
discussion  with  unit  leadership.  Part  of  the  art  of  the  AAR  is  determining  how  far  and  how  long 
to  let  the  discussion  wander  from  planned  focus  areas.  Weighing  heavily  into  this  decision  is  an 
assessment  by  the  facilitator  and  unit  leadership  as  to  the  derived  benefit  to  training  objectives 
and  ultimate  unit  performance  of  the  current  discussion. 

Concern  over  wandering  discussion  should  be  limited.  Unit  professionalism  tends  to 
serve  as  an  invisible  hand  to  keep  open  discussions  focused  on  real  improvement.  Facilitators 
should  favor  free  discussion,  but  when  the  facilitator  deems  that  the  discussion  has  become  non¬ 
productive,  he/she  should  use  open-ended  questions,  such  as  how  the  discussion  pertains  to 
specific  training  objectives,  to  subtlety  draw  the  discussion  back  to  desired  focus  areas,  as 
opposed  to  blatantly  shutting  down  or  shifting  discussion. 

Follow-on  questions  are  particularly  important  when  the  AAR  is  focusing  on  cognitive 
tasks,  as  the  “ground  truth”  consists  of  what  was  sensed  and  understood  by  individuals,  what 
perception  was  communicated,  and  the  decisions  based  upon  that  communication.  This 
cognitive  “ground  truth”  will  likely  be  not  apparent  from  sensor  network  information,  though 
recorded  communications  probably  provides  the  most  direct  access. 

In  the  AAR,  as  the  participants  identify  specific  issues,  the  facilitator  expedites  the  unit’s 
identification  of  unit  changes  to  resolve  issues.  Importantly,  the  facilitator  expedites  assignment 
of  responsibility  for  executing  the  change  and  a  schedule  for  executing  that  change.  The 
facilitator  and  training  unit  should  consider  this  a  contract  for  improvement.  Review  of  this 
contract,  including  review  of  issues,  unit  plans  to  resolve  those  issues,  and  individual 
responsibilities  within  those  plans,  is  an  effective  manner  to  close  an  AAR. 

AAR  Follow-Up 

Immediately  after  the  AAR,  the  facilitator  meets  with  his  counterpart  to  review  the 
execution  of  the  AAR  and  AAR  results.  This  is  an  opportunity  to  refine  the  AAR  generated 
contract,  including  mutually  clarifying  any  points  of  confusion.  This  is  also  an  opportunity  for 
the  facilitator  to  discuss  the  execution  of  the  AAR  with  his/her  counterpart,  including  identifying 
methods  to  improve  the  next  AAR. 

The  facilitator  also  reviews  AAR  results  with  fellow  facilitators,  especially  superiors. 
Facilitators  review  their  performance.  They  also  look  for  cross-organizational  trends.  This 
review  might  lead  to  changes  to  facilitator  plans  or  changes  to  the  training  event. 

Finally,  facilitators  save  AAR  products  for  future  use  within  the  training  event  and  within 
future  training  events.  Products  include  prepared  open-ended  questions,  information 
presentations,  and  examples  of  how  units  performed  various  tasks.  These  examples  are  sanitized 
to  remove  any  reference  to  a  specific  unit.  They  then  become  “a- ways”  or  performance  models 
for  future  AARs. 
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APPENDIX  C:  Acronym  List 


AAIRS-  After  Action  Intelligent  Review  System 

AAR  -  After  Action  Review 

ADA  -  Air  Defense  Artillery 

AKO  -  Army  Knowledge  Online 

BC  -  Battle  Commander 

BCTP  -  Battle  Command  Training  Program 

CACCTUS  -  Combined  Arms  Command  and  Control  Trainer  Upgrade  System 

CALL  -  Center  for  Army  Lessons  Learned 

COI  -  Community  of  Interest 

CRM  -  Customer  Relations  Management 

CTCs  -  Combat  Training  Centers 

DAR  -  During  Action  Review 

DIS  -  Distributed  Interactive  Simulation 

DIVAARS  -  Dismounted  Infantry  Virtual  After  Action  Review  System 

ELO  -  Enabling  Learning  Objectives 

EXCON  -  Exercise  Control 

FA  -  Functional  Area 

ISD  -  Instructional  Systems  Design 

IT AAR  -  Integrated  Theory  of  AAR 

ITS  -  Intelligent  Tutoring  Systems 

JELC  -  Joint  Exercise  Life  Cycle 

JFCOM  -  US  Joint  Forces  Command 

JRTC  -  Joint  Readiness  Training  Center 

JTS  -  Joint  Training  System 

LTS  -  Long-  Term  Memory  Storage 

MDMP  -  Military  Decision  Making  Process 

NTC  -  National  Training  Center 

O/C  -  Observer  Controller 

O/CT  -  Observer  Controller  Trainer 

OPFOR  -  Opposition  Forces 

POI  -  Programs  of  Instruction 
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SA  -  Situation  Awareness 

SECI  -  Socialization,  Extemalization,  Combination  and  Internalization 

S02  -  Study  of  Organizational  Opinion 

STX  -  Situation  Training  Exercise 

TLO  -  Terminal  Learning  Objective 

TLP  -  Team  Leading  Procedures 

TRADOC  -  U.S.  Army  Training  and  Doctrine  Command 
TTP  -  Tactics,  Techniques,  and  Procedures 
USASMA  -  U.S.  Army  Sergeants  Major  Academy 
VIRTE  -  Virtual  Technology  and  Environments 
WLC  -  Warrior  Leaders  Course 
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